how to install Airflow on Kubernetes | Run Airlfow on Kubernetes cluster locally скачать в хорошем качестве

how to install Airflow on Kubernetes | Run Airlfow on Kubernetes cluster locally

In this video we are installing Apache Airflow on a Kubernetes cluster. This is a Quick Guide guide on the installation process.🏗️ Kubernetes offers scalability, flexibility, and better resource management for your workflows. This was commonly requested after the Airflow's installation series. Here’s a high-level approach to get started: 1️⃣ Use Helm – Deploy Airflow with the official Helm chart for easy setup. 2️⃣ Use Kind - To create a Kubernetese cluster 3️⃣ Use Kubectl – To manage the Kubernetese cluster and resources. Kubernetes + Airflow = Robust, Scalable Workflow Orchestration! 💡 ⬇️ #ApacheAirflow #Kubernetes #dataengineering Link to required tools: Kubectl: https://kubernetes.io/docs/tasks/tools/ Helm: https://helm.sh/docs/intro/install/ Kind: https://kind.sigs.k8s.io/ Docker Desktop: https://www.docker.com/products/docke... How to build Airflow custom image:    • How to install Apache Airflow on Docker? |...   Various commands used in this video to create different objects. kind create cluster --name airflow-cluster kind get clusters helm repo add apache-airflow https://airflow.apache.org helm install dev-release apache-airflow/airflow --namespace airflow kubectl get pods --namespace airflow helm list --namespace airflow kubectl delete --all pods --namespace=airflow docker build --pull --tag airflow-dags:0.0.1 . kind load docker-image airflow-dags:0.0.1 --name airflow-cluster helm upgrade dev-release apache-airflow/airflow --namespace airflow --set images.airflow.repository=airflow-dags --set images.airflow.tag=0.0.1 kubectl port-forward svc/dev-release-webserver 8080:8080 --namespace airflow localhost:8080