Amazon Kinesis Information Analytics is a totally managed service for Apache Flink that reduces the complexity of constructing, managing, and integrating Apache Flink functions with different AWS companies. Apache Flink is an open-source framework and engine for stateful processing of information streams. It’s extremely accessible and scalable, delivering excessive throughput and low latency for probably the most demanding stream-processing functions.

On this publish, we spotlight three new movies so that you can be taught extra about Apache Flink and Kinesis Information Analytics, together with open-source contributions to Apache Flink, our learnings from working hundreds of Flink jobs on a managed service, and the way we use Kinesis Information Analytics and Apache Flink to allow machine studying (ML) in Alexa.

In Introducing the brand new Async Sink, we current the brand new Async Sink framework, an open-source contribution to make it simpler than ever to construct sink connectors for Apache Flink. You possibly can be taught in regards to the want for the Async Sink framework and the way we constructed it, adopted by a demo of constructing a brand new sink to Amazon CloudWatch to ship CloudWatch metrics, in below 20 minutes! The Async Sink framework bootstraps improvement of Flink sinks, is appropriate with Apache Flink 1.15 and above, and has already seen utilization by the group past constructing new sinks to AWS companies.

The video Sensible learnings from working hundreds of Flink jobs shares perception from working Kinesis Information Analytics, a managed service for Apache Flink that runs tens of hundreds of Flink jobs. You possibly can be taught classes primarily based on our expertise of working Apache Flink at very massive scale, referring to points equivalent to out-of-memory errors, timeouts, and stability challenges. The video additionally covers bettering software efficiency with reminiscence tuning and configuration adjustments and the approaches to automating job well being monitoring and administration of Flink jobs at scale.

“Alexa, be quiet!” Finish-to-end near-real time mannequin constructing and analysis in Amazon Alexa discusses how Alexa has constructed an automatic end-to-end answer for incremental mannequin constructing or fine-tuning ML fashions by means of steady studying, continuous studying, or semi-supervised lively studying. Alexa makes use of Apache Flink to rework and uncover metrics in actual time. On this video, you find out about how Alexa scales infrastructure to satisfy the wants of ML groups throughout Alexa, and discover particular use instances that use Apache Flink and Kinesis Information Analytics to enhance Alexa experiences to please clients.

To be taught extra about Kinesis Information Analytics for Apache Flink, go to our product web page.

In regards to the writer

Deepthi Mohan is a Principal Product Supervisor on the Kinesis Information Analytics staff.