I need to create a POC with Step Functions

Question

I need to create a POC to implement an ETL process with SAS tables from a Glue job without business rules with 7 source tables adding another Job with business rules in each table and after the ETL imputs the table destination in S3 in csv or parquet format, this entire process must be configured in AWS Step Functions.

Basel Mohamed · Accepted Answer

This POC outlines how to set up an ETL process using AWS Glue and Step Functions. 
You will:
Create Glue jobs to extract and transform data.
Configure a Step Functions workflow to manage the ETL process.
Set up the necessary AWS resources, including S3 buckets and IAM roles.
Run and monitor the ETL process using Step Functions.
This setup provides a foundational framework for an ETL process, which you can expand with additional steps, error handling, and more complex business rules as needed.

Set Up AWS Glue Jobs
Glue Job 1: Extract Data from SAS Tables
Create an S3 Bucket:

Set up an S3 bucket to store the extracted data.
Create a Glue Job:

Use AWS Glue to read SAS tables from the source and write the data to the S3 bucket in Parquet format.

I need to create a POC with Step Functions

Relevant content