Mastering Change Data Capture With DLT скачать в хорошем качестве

Mastering Change Data Capture With DLT

Transactional systems are a common source of data for analytics, and Change Data Capture (CDC) offers an efficient way to extract only what’s changed. However, ingesting CDC data into an analytics system comes with challenges, such as handling out-of-order events or maintaining global order across multiple streams. These issues often require complex, stateful stream processing logic. This session will explore how DLT simplifies CDC ingestion using the Apply Changes function. With Apply Changes, global ordering across multiple change feeds is handled automatically — there is no need to manually manage state or understand advanced streaming concepts like watermarks. It supports both snapshot-based inputs from cloud storage and continuous change feeds from systems like message buses, reducing complexity for common streaming use cases. Talk By: Jacob Gollub, Software Engineer, Square ; Ray Zhu, Director, Product Management, Databricks Here’s more to explore: Production ready data pipelines for analytics and AI: https://www.databricks.com/solutions/... The Big Book of Data Engineering: https://www.databricks.com/resources/... See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dat... Connect with us: Website: https://databricks.com Twitter:   / databricks   LinkedIn:   / databricks   Instagram:   / databricksinc   Facebook:   / databricksinc