DataFusion - The Database Building Toolkit (with Andrew Lamb) скачать в хорошем качестве

DataFusion - The Database Building Toolkit (with Andrew Lamb)

Building a database is a serious undertaking. There are just so many parts that you have to implement before you even get to a decent prototype, and so many hours of work before you could begin working on the ideas that would make your database unique. Apache DataFusion is a project that hopes to change all that, but building an extensible, composable toolkit of database pieces, which could let you build a viable database extremely quickly, and then innovate from that starting point. And even if you’re not building a database, it’s a fascinating project to explain how databases are built. Joining me to explain it all is Andrew Lamb, one of DataFusion’s core contributors, and he’s going to take us through the whole stack, how it’s built and how you could use it. Along the way we cover everything from who’s building interesting new databases and how you manage a large, open-source Rust project. – DataFusion Homepage: https://datafusion.apache.org/ DataFusion on Github: https://github.com/apache/datafusion DataFusion Architecture (with diagrams!):    • Apache Arrow DataFusion Architecture Part 1   Datalog: https://docs.racket-lang.org/datalog/ Tokio: https://tokio.rs/ Andrew’s Homepage: http://andrew.nerdnetworks.org/ Andrew’s Blog Post about Tokio: https://thenewstack.io/using-rustlang... Velox: https://velox-lib.io/ Arroyo: https://www.arroyo.dev/ Synnada: https://www.synnada.ai/ LanceDB: https://lancedb.com/ SDF+DBT: https://docs.sdf.com/integrations/dbt... Support Developer Voices on Patreon:   / developervoices   Support Developer Voices on YouTube:    / @developervoices   Kris on Bluesky: https://bsky.app/profile/krisajenkins... Kris on Mastodon: http://mastodon.social/@krisajenkins Kris on LinkedIn:   / krisjenkins   – 0:00 Intro 3:14 What is DataFusion? 10:49 DataFusion's Feature List 19:16 What Would I Add To DataFusion To Create A New DB? 24:28 How Would I Teach DataFusion About A Custom Index? 31:29 Pipelining & Data Streaming 42:07 How Is DataFusion Implemented? 57:43 Building A Big Rust Open Source Project 1:09:25 How Would I Extend DataFusion Beyond The Shipped Featureset? 1:16:20 Who Are The Most Interesting Users Of DataFusion? 1:23:47 What's Andrew Working On Next? 1:30:12 Outro