Visual Flow - the future of low-code data processing скачать в хорошем качестве

Visual Flow - the future of low-code data processing

The video covers how to use a low-code platform called Visual Flow on Apache Spark to perform ETL (Extract, Transform, Load) operations on data. The architect of Visual Flow demonstrates how to create a workflow in Visual Flow to ingest data from different sources, apply transformations, and store the results in a data warehouse using Spark. Key takeaways: Apache Spark is a powerful distributed computing framework that can be used for ETL operations. Visual Flow works on top of Spark to provide a user-friendly interface for building ETL workflows that can scale to handle large datasets. Visual Flow can help democratize data engineering by making it accessible to a wider range of users, including those without a background in coding or computer science. Visual Flow can help simplify this process and make Spark more accessible to users with less technical expertise. Timecodes: 00:00 - Introduction 00:44 - What is Visual Flow 01:08 - Why should you consider Visual Flow for your ETL project? 02:30 - Visual Flow history 04:15 - How can I use Visual Flow? 04:48 - Manual coding (ETL on Scala) vs Visual Flow comparison 05:43 - How Visual Flow works 06:35 - Supported data sources 06:57 - Visual Flow user roles 07:37 - Visual Flow parameter management 08:13 - Visual Flow IDE 08:52 - Data pipelines in Visual Flow 09:28 - Visual Flow demonstration and UI review 21:23 - Summary 22:09 - Visual Flow Services