FREE COURSE: How to Build Scalable Data Pipelines with Kestra on GCP скачать в хорошем качестве

FREE COURSE: How to Build Scalable Data Pipelines with Kestra on GCP

Workflow orchestration is one of the most important skills in data engineering. And also one of the hardest to get right in production. In this free full course video, you’ll learn how to build scalable, reliable workflows with Kestra on Google Cloud Platform (GCP). We start locally with Docker and move step by step toward real production-style pipelines using Cloud Storage (GCS), BigQuery, and event-driven triggers. This is a hands-on, end-to-end walkthrough. You’ll learn how Kestra workflows (flows) are defined in YAML, how tasks exchange inputs and outputs, and how to design pipelines that stay maintainable as they grow. Throughout the course, you’ll: ▶️ Run real Python and Java code inside Kestra workflows ▶️ Synchronize workflows and code from GitHub using Namespace & Tenant Sync ▶️ Build reusable pipelines with subflows ▶️ Load and transform data with BigQuery + GCS ▶️ Process multiple files dynamically using loops ▶️ Scale execution with parallel tasks ▶️ Implement error handling, fallbacks, and retries ▶️ Build event-driven pipelines triggered by file uploads (Pub/Sub) ▶️ Monitor, debug, replay workflows, and build dashboards with execution metrics If you found this useful, leave a like or a comment! It helps the channel grow and tells me what to cover next.