Using Windows to Aggregate Data with the Apache Flink® Table API | Apache Flink® Table API скачать в хорошем качестве

Using Windows to Aggregate Data with the Apache Flink® Table API | Apache Flink® Table API

Get started: https://cnfl.io/apache-flink-table-ap... | In this video, we'll show how to implement windows using the Apache Flink Table® API. We'll break down the problems windows are trying to solve and show how the different window types approach the solution. Real-time data streaming differs from batch-oriented processes by focusing on individual events and consuming them as they happen. However, this creates complications when we perform a calculation that operates over many events rather than just one. In that case, streaming tools group events into time-based windows. However, even though the windows contain multiple events, they are still processed one at a time. It is the result of the aggregation that gets rolled up into the window. The aggregations supported by the Flink Table API have optimizations that allow them to accumulate results with a minimal amount of state. An average aggregation normally requires keeping track of each value. However, it can be optimized to track only the sum of the values and the total number of records. This reduces the large list to just two individual numbers. The Table API also supports different types of windows, such as tumbling and sliding. These are similar but differ in key ways. Tumbling windows are equally spaced with zero overlap, whereas sliding windows, though still equal, usually have some overlap. Where each of them is used depends on the specific use case. RELATED RESOURCES ►GroupBy Window Aggregation - https://bit.ly/3FzCya8 ►Tumbling Windows - https://bit.ly/4bG7d1y ►Sliding Windows - https://bit.ly/3Fkvqyu CHAPTERS 00:00 - Intro 00:39 - What are the problems with handling events in batches? 01:42 - Why is data streaming superior to batch processing? 02:22 - How can windows allow us to stream data, but still aggregate over time? 03:34 - What is the structure of a windowed query? 03:59 - How can we implement a tumbling window? 06:17 - How can we implement a sliding window? 08:07 - Closing CONNECT Subscribe, if you dare:    / @confluentdeveloper   Community Slack: confluentcommunity.slack.com X: https://x.com/confluentinc Linkedin:   / confluent   GitHub: https://github.com/confluentinc Site: https://developer.confluent.io ABOUT CONFLUENT DEVELOPER Confluent Developer provides comprehensive resources for developers looking to learn about Apache Kafka®, Apache Flink®, Confluent Cloud, Confluent Platform, and any other technology related to the broader Data Streaming Platform. Content on Confluent Developer includes courses, getting started guides, topical deep-dives, patterns, tutorials, and listings of community events. Learn more at https://developer.confluent.io. #apacheflink #flink #confluent