The Data Virtualization Evolution: Presto -- AWS Athena | David Krakov, Varada CTO | varada.io скачать в хорошем качестве

The Data Virtualization Evolution: Presto -- AWS Athena | David Krakov, Varada CTO | varada.io

Data virtualization was born to empower data consumers to query any data on S3 without moving it. It has quickly become synonymous with reducing data deduplication and storage consumption. It also significantly lowers the cost of both data-ops and devops. This is especially true on the cloud, where data lakes like S3 are a great match to decoupled query engines and data virtualization technologies. One of the early query engines developed to support high-performance data virtualization is Presto, which was born in Facebook labs to replace Hive. As a versatile distributed query engine it can support a wide range of data sources. With no data duplication, Presto gives data teams centralized access controls, and leverages a shared catalog to make collaboration easier. Among query engines, Presto stands out because of its broad support, deep extensibility, and powerful standard ANSI SQL language. Presto also integrates a basic cost-based optimizer that evaluates different execution plans for each query and chooses the best one based on data statistics and available resources, reducing overall processing time. AWS Athena is a critical building block in any modern data lake architecture. And indeed, Athena works exceptionally well out of the box until users run into performance issues. Even though core Presto has powerful tools for optimization, a zero devOps solution such as Athena doesn’t include any tooling for analyzing performance issues. The result is that when Athena deployment gains adoption in the organization, users run into roadblocks trying to productionalize the system which often amounts to massive investment in ETL optimization for data before consumption. While the benefits of using Athena still outweigh the limitations, these issues are what ultimately lead organizations to abandon the pure data lake strategy and adopt a hybrid approach, duplicating data into a data warehouse. Instead of being the unifying solution promised by data virtualization, Athena and the data lake ends up being relegated to yet another data silo. *** 🌐Check Out Our Website! https://varada.io ✿ Social Links: LinkedIn:   / varadaio   Twitter:   / varadaio   🔔 SUBSCRIBE IF YOU ARE NEW AND TURN THE BELL ON Watch More Videos: ▶️ Leverage the Power of Big Data Index to Dramatically Accelerate Queries on AWS Athena    • The Data Virtualization Evolution: Presto ...   ▶️ VARADA | Big Data Small Effort    • Video   ▶️ Varada Data Virtualization Platform    • Varada Data Virtualization Platform | vara...   Thanks for being with us! 👍 Like the video 💬 Comment if you made it till the end! 🔗 Share the video with anyone you think it might help :) "