Stop Manual Tuning: Predictive Optimization in Databricks Explained скачать в хорошем качестве

Stop Manual Tuning: Predictive Optimization in Databricks Explained

Predictive Optimization at Scale: 2025 Innovations & What's Next in 2026! Learn how Databricks is fully automating lakehouse performance and storage efficiency with Predictive Optimization (PO). In this video, we break down the major milestones PO achieved in 2025 and give you a sneak peek into the highly anticipated features dropping in 2026. Enabled by Default: Predictive Optimization is now the default intelligence layer for all new Unity Catalog managed tables, helping users save tens of millions of dollars by automatically vacuuming exabytes of unreferenced data. Automatic Statistics: Experience up to 22% faster queries as PO automatically maintains accurate statistics based on your query behavior, eliminating the need for manual ANALYZE commands. Faster & Cheaper VACUUMs: A new log-based approach bypasses costly directory listings, making VACUUM execution up to 6x faster and 4x cheaper. Automatic Liquid Clustering: PO now analyzes your query workloads and tests different clustering keys to automatically apply the best data layout for optimal performance. Platform-Wide Coverage: PO now natively supports both Delta and Iceberg tables, as well as Materialized Views and Streaming Tables via Lakeflow Spark Declarative Pipelines. What’s Coming in 2026? Auto-TTL (Automatic Row Deletion): You will be able to set simple "time-to-live" policies on any Unity Catalog managed table, and PO will automatically handle the soft-deletes and physical vacuuming of expired rows. Enhanced Observability: Track the exact ROI of PO in the new Data Governance Hub, featuring clear visualizations of bytes compacted, clustered, and vacuumed, along with estimated storage cost savings. For more articles: https://www.nextgenlakehouse.com/ Author: Youssef Mrini