Apache Iceberg v3 - Evolving the Open Table Standard for the Phase of the Iceberg Lakehouse скачать в хорошем качестве

Apache Iceberg v3 - Evolving the Open Table Standard for the Phase of the Iceberg Lakehouse

This session walks through how Apache Iceberg has become the foundational table format of the modern lakehouse—and how v3 pushes it into a new phase. We start by grounding everyone in the lakehouse architecture: separating storage, processing, table format, and catalog so any engine can work with the same governed data without creating new silos. From there, we trace Iceberg’s evolution from v1’s basic snapshot-based metadata and copy-on-write behavior, to v2’s merge-on-read with position and equality deletes, and into v3’s major advancements: deletion vectors for faster updates, row-level lineage for better troubleshooting and auditability, default column values to smooth schema evolution, and richer data types like variant, geometry, and geography that unlock more complex analytics and AI use cases. Early work on encryption and portable security models rounds out the picture, making Iceberg more capable as a shared enterprise standard. We then look ahead at where the community is taking Iceberg next. The talk covers active v4 design discussions around faster commits for streaming workloads, smarter metadata (including Parquet-based columnar metadata), more portable table definitions via relative paths, and a more extensible “files API” to support emerging storage formats without rewriting core logic. Along the way, you’ll see how these innovations reduce infrastructure costs, improve performance, and strengthen governance—while making it easier for multiple engines, catalogs, and AI systems to share one consistent view of your data. The session closes by connecting these roadmap themes back to Dremio’s NextGen Cloud and ecosystem, showing how Iceberg’s open evolution underpins a more flexible, future-ready lakehouse.