Data Lakehouse with dbt and DuckLake скачать в хорошем качестве

Data Lakehouse with dbt and DuckLake

Everybody loves DuckDB, and the community is hyped with DuckLake being released, but how do you actually implement an end-to-end solution for running your ETL pipelines on top of DuckDB and DuckLake? In this video, we'll show you how to use dbt, through the dbt-duckdb adapter, as the glue that makes it all possible, covering the whole process, from installing and setting up the tech stack, to data ingestion, transformation and exporting. You'll also be able to follow along by cloning our Data Lab project on GitHub (link below), or use it to scaffold a project for your own needs. ► Note: In future videos, we'll continue building on top of Data Lab, and we have planned at least another video where we load the data we prepared in this video into KùzuDB and run a few interesting computations. I hope you find this to be useful, or at least fun or interesting! If have you have any questions or suggestions, please leave a comment below or link up on social media. 🔗 Resources: 💻 GitHub repo: https://github.com/DataLabTechTV/datalab 📝 Blog post: https://datalabtechtv.com/posts/data-... 🔔 Subscribe for more deep dives data science, data engineering, and all things data! ☁️ Follow the channel on Bluesky: https://bsky.app/profile/datalabtecht... 🎧 Join our community's Discord:   / discord   👽 Join our subreddit:   / datalabtechtv   💻 Check out the channel's GitHub: https://github.com/DataLabTechTV 🌐 Visit our website for unedited research notes: https://datalabtechtv.com #dataengineering #ducklake #dbt #database #datalakehouse #duckdb #minio #etl #datalab 00:00:00 Intro 00:00:50 Use Case 00:01:19 Tech Stack 00:07:58 Storage Layout 00:10:54 Workflow 00:12:05 Introducing Data Lab 00:12:43 Quick Start 00:17:44 Data Ingestion 00:25:19 Data Transformation 00:28:22 dbt Configuration 00:31:27 dbt Macros 00:33:40 dbt Models 00:35:49 DuckDB List Functions 00:39:00 dbt Tests 00:42:45 dbt Documentation 00:44:13 DuckLake Access 00:46:30 Catalog Backup 00:48:55 Data Exporting 00:50:07 Outro