End-to-end Data Engineering & Machine Learning platform built Databricks Databricks Capstone project скачать в хорошем качестве

End-to-end Data Engineering & Machine Learning platform built Databricks Databricks Capstone project

Databricks Capstone Project submission for the Databricks AI challenge-2 sponcered by ​‪@Databricks‬ organised by ‪@codebasics‬ & ‪@Indian_Data_Club‬ UrbanRide(Not real) Lakehouse Data Platform is an end-to-end data engineering project that simulates the data infrastructure of a mid-scale ride-hailing company operating across major cities in India. Problem Statement : Imagine you run a ride-hailing company. You have 89,000 customers. 19 million payment transactions. Millions of trips across 5 cities. Hundreds of thousands of app events every single day. And you cannot answer a single one of these questions: Which of my customers are about to leave — and who should I call first? Are any of my 19 million payments fraudulent — and which ones? How many drivers do I need in Delhi next Friday? Who are my best customers — and what makes them different from everyone else? That was UrbanRide's reality. Enormous amounts of data. Zero visibility. Zero intelligence. This project Explains how I solved that — from raw files sitting in a folder, to a fully automated machine learning platform running on Databricks every single night. Every design decision I made — I'll explained why I made it, what alternatives exist, and what would break if I made a different choice. Like Share Subscribe !!! 00:00 intro 00:34 problem statement 04:28 data exploration 07:12 bronze layer 14:42 silver layer 21:37 gold layer 33:11 model building 53:00 jobs and pipelines and dashboards Git hub link : https://github.com/vishalbankar/Urban... #databricks #ai #project #capstoneproject #dataengineering #codebasics #DataWithIDC