BRISTECH BYTESIZE #18 - Kevin Kho - Data Flows with Prefect скачать в хорошем качестве

BRISTECH BYTESIZE #18 - Kevin Kho - Data Flows with Prefect

Kevin Kho - An Intro to Workflow Management with Prefect As data pipelines become increasingly complex and interconnected, workflow management systems are being used to schedule and monitor tasks. Prefect is an open source workflow management system designed for large-scale data processes. In this talk, we’ll show how to get started with Prefect to orchestrate data validation workflows to preserve data integrity before it’s used in downstream processes. About Kevin Kho - Open Source Community Engineer at Prefect Kevin works for Prefect, an open source workflow orchestration management system. Previously, he was a data scientist at Paylocity, where he worked on adding machine learning features to their Human Capital Management (HCM) suite. Outside of work, he is a contributor for Fugue, an abstraction layer that lets users port Python and SQL code to Pandas, Spark and Dask. He also organises the Orlando Machine Learning and Data Science Meetup. Panel discussion with... Richard Pelgrim - Data Science Evangelist @ Coiled Jared Rhizor - Lead Engineer @ Airbyte