Glue ETL Job Failling

0

I' m trying to follow a tutorial about Glue, to create a Data Lake from a Mysql Database (RDS). Unfortunately i'm stucked in some issues, and i can not proceed with my learning.

I've used a Blueprint in glue to create the ETL process. This created a crawller and some jobs. The crawller seems to be ok, but the jobs no, the error is: "failed to execute with exception Internal service error: Invalid Input Provided".

Enter image description here

Investigating one of ETL Jobs, i don't have permissions to change it, I receive the following error: "[s3.amazonaws.com] putObject: AccessDenied: Access Denied". If i browse the S3 from glue i can see the bucket, but from s3 main page i can not.

Even with my root i can not change any permission: Enter image description here

The ETL Job error could be related with the s3 permissions error or should be other? Any tip?

Thanks and best regards,

Ricardo

asked 2 years ago457 views
1 Answer
0

It seems like you're encountering permission issues with your Glue ETL jobs and S3 bucket. Let's address each problem separately.

Invalid Input Provided Error: This error message usually indicates that there is an issue with the configuration or input provided to the ETL job. To troubleshoot this, you can check the following: Make sure the source and target connections in your ETL job are correctly configured. Verify that the input data format and schema match the expected format for the job. Double-check any parameters or settings that you've provided to the job and ensure they are valid. If you've confirmed that the inputs are correct, you can try creating a new ETL job from scratch and see if the issue persists. If it does, you might need to review the logs and error messages for more specific details on what is causing the problem.

S3 Access Denied Error: The "Access Denied" error when trying to change permissions on the S3 bucket suggests that you don't have the necessary IAM (Identity and Access Management) permissions to perform those actions. To address this: Ensure that you have the appropriate IAM permissions to access and modify the S3 bucket. You can check your IAM policies to confirm if they grant the necessary permissions. If you're using an IAM user account, contact the AWS account administrator or an IAM administrator to grant you the necessary permissions. Make sure that the AWS account you're using has the necessary permissions to access the S3 bucket. Additionally, if you can see the S3 bucket from the Glue console but not from the main S3 page, it could be due to different IAM roles being used for the Glue service and your main AWS account. The Glue service might be using a specific IAM role with different permissions.

In summary, you should investigate and resolve the S3 access issue first, as it could be causing the ETL job error. Once you have the appropriate permissions to access the S3 bucket, you can retry your ETL job and see if the "Invalid Input Provided" error persists. If it does, review your job configuration and inputs to ensure they are correct

answered 9 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions