Data Processing for Any Data and Any Scale скачать в хорошем качестве

Data Processing for Any Data and Any Scale

Your data pipeline wasn't built for JPEGs, PDFs, and Zoom recordings. Jay Chia   / chiajay   (co-founder of Daft) shows why traditional ETL breaks when you're downloading a million URLs, decoding them as images, and running GPU models on every single one. In this talk, Jay walks through what happens when your "simple" data processing job inflates from 1TB of URLs to 20TB of decoded images—and why Spark makes you manually partition everything just to avoid crashing. See how modern data pipelines handle the messy reality of multimodal data: rate limits from model providers, GPU scheduling, and that one failed API call at row 900,000 that ruins your entire run. Watch a live demo of Daft turning ImageNet URLs into embeddings in four lines of code, with automatic batching that keeps your memory from exploding and your GPUs busy. Learn why processing videos and images is more like running a production web app than a SQL query—your pipeline needs to handle user-defined functions that might do anything from running CLIP to calling OpenAI a million times. For engineers who've ever watched their cluster die trying to process images, teams building data infrastructure for computer vision, or anyone who thinks "just use Spark" until they actually try to run PyTorch models at scale. Data Systems for AI: Multimodal Madness, hosted by Eventual Computing (the team building Daft). — Daft. Simple and reliable data processing for any modality and scale. Explore → https://daft.ai/ Build → https://docs.daft.ai/ Connect → https://www.daft.ai/slack Contribute or ⭐️ → https://github.com/Eventual-Inc/Daft Learn → https://daft.ai/blog pip install daft