Скачать с ютуб Data Con LA 2022 - Building a serverless data processing pipeline with PySpark on cloud в хорошем качестве

Data Con LA 2022 - Building a serverless data processing pipeline with PySpark on cloud

Suman Debnath, Principal Developer Advocate(Data Engineering), Amazon Web Services Data is all over the place, and what matters is how we manage that data and make sense out of it and take some meaningful data driven decision. In this session we will discuss about whole data engineering pipeline, starting from data collection, processing, analysis and visualization in a complete serverless fashion. We will pick some opensource dataset and shall store and process it on cloud(AWS). While the focus would be more on the general understanding of data pipeline aspects of data engineering, but during the process we will learn few of the AWS services which can help us to achieve our goal in an effective and efficient way.