Bridging Big Data and Machine Learning Ecosystems: A Cloud Native Approac... Johnu George & Shiv Jha скачать в хорошем качестве

Bridging Big Data and Machine Learning Ecosystems: A Cloud Native Approac... Johnu George & Shiv Jha

Don't miss out! Join us at our next Flagship Conference: KubeCon + CloudNativeCon North America in Atlanta (November 10-13). Connect with our current graduated, incubating, and sandbox projects as the community gathers to further the education and advancement of cloud native computing. Learn more at https://kubecon.io Bridging Big Data and Machine Learning Ecosystems: A Cloud Native Approach Using Kubeflow - Johnu George & Shiv Jha, Nutanix In today's data-driven landscape, bridging the gap between scalable big data systems (e.g., Apache Spark, Iceberg) and machine learning frameworks (e.g., PyTorch) while minimizing data movement and serialization overhead is a critical challenge. Traditional workflows require costly data serialization between storage (e.g., Parquet/Iceberg) and training frameworks, creating bottlenecks leading to inefficient resource utilization in distributed training. This talk explores a cloud-native solution using Kubeflow for end-to-end ML orchestration and Apache Arrow for high-performance data interchange, enabling seamless integration of analytics and ML workflows.