Performing ETL on a Dataset by Using AWS Glue скачать в хорошем качестве

Performing ETL on a Dataset by Using AWS Glue

Lab overview and objectives: Big data problems often involve a large number of heterogeneous data sources. As a data analyst, you might not know the schema for some data sources. This is the variety aspect of the five Vs of big data (volume, variety, velocity, veracity, and value). In this lab, you will work with AWS Glue to perform extract, transform, and load (ETL) for a dataset. You can direct AWS Glue to a data source, and it can infer a schema based on the data types that it discovers. Then, AWS Glue builds a Data Catalog that contains metadata about the various data sources. AWS Glue is similar to Amazon Athena in that the actual data that you analyze remains in the data source. The key difference is that you can build a crawler with AWS Glue to discover the schema and then extract the data from the dataset. You can also transform the schema and then load the data into an AWS Glue database. You can then analyze the data by using SQL statements in Athena. In this lab, you will learn how to use AWS Glue to import a dataset from Amazon Simple Storage Service (Amazon S3). You will then extract the data, transform its schema, and load the dataset into an AWS Glue database for later analysis by using Athena. After completing this lab, you will be able to do the following: Access AWS Glue in the AWS Management Console and create a crawler. Create an AWS Glue database with tables and a schema by using a crawler. Query data in the AWS Glue database by using Athena. Create and deploy an AWS Glue crawler by using an AWS CloudFormation template. Review an AWS Identity and Access Management (IAM) policy for users to run an AWS Glue crawler and query an AWS Glue database in Athena. Confirm that a user with the IAM policy can use the AWS Command Line Interface (AWS CLI) to access the AWS Glue database that the crawler created. Confirm that a user can run the AWS Glue crawler when source data changes.