Beyond Tiered Storage: Deep dive into WarpStream's Storage Engine (Richie Artoul) | Current 24 скачать в хорошем качестве

Beyond Tiered Storage: Deep dive into WarpStream's Storage Engine (Richie Artoul) | Current 24

More: https://current.confluent.io | In this talk, we'll discuss in technical detail how to take the concept of tiered storage to its logical extreme by building an Apache Kafka protocol compatible system that has zero local disks. In this talk we'll cover how to design a storage engine from the ground up around commodity object storage. We'll dive deep into WarpStream's storage engine implementation and cover topics like: ingestion, metadata, compaction, fetch, and service discovery. Finally, at the end we'll see how the concept of tiered storage can be re-introduced even in a system that has zero local disks to make different trade-offs around cost and performance. LEARN MORE ► Confluent Developer: https://developer.confluent.io CONNECT Subscribe on YouTube:    / @confluentdeveloper   Community Slack: confluentcommunity.slack.com X: https://x.com/confluentinc Linkedin:   / confluent   GitHub: https://github.com/confluentinc Site: https://developer.confluent.io ABOUT CONFLUENT DEVELOPER Confluent Developer provides comprehensive resources for developers looking to learn about Apache Kafka®, Apache Flink®, Confluent Cloud, Confluent Platform, and any other technology related to the broader Data Streaming Platform. Content on Confluent Developer includes courses, getting started guides, topical deep-dives, patterns, tutorials, and listings of community events. Learn more at https://developer.confluent.io. #apachekafka #apacheflink #confluent