Access Denied on S3 from AWS Glue

Question

Hi,
 I am getting AccessDenied Error on S3, while trying to write to S3 with AWS Glue.

Below is the error message:
com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied;

Below are the Glue Job details:
1.	AWS Glue reads from SAP Hana table and tries to write the data to S3 bucket.
2.	The Glue Job role has full access to S3 bucket.
For troubleshooting, I created another aws glue job which reads a file from S3 bucket and writes to same bucket in another folder. And this works fine. But when I read from Sap database, it throws AccessDenied error.
Please advise.

Didier_Durand · Answer

Hi,

Maybe Glue accesses SAP via another role and doesn't switch properly to your execution role for S3 access.

To exactly see what's happening in your setup, I would suggest to leverage CloudTrail: you'll will be able to see all APIs calls. In particular, you will see the failing one for S3 and see under which identity it is made to check the hypothesis above.

This KB article will show you how to do that: https://repost.aws/knowledge-center/cloudtrail-track-api

Best,

Didier

kranthi putti · Answer

Hello,

It seems like your AWS Glue job role might not be correctly switching to the necessary role for accessing S3 when reading from SAP HANA. Here are the steps to troubleshoot:

**Verify IAM Role Permissions:**
Ensure the IAM role for the Glue job has the necessary S3 permissions (s3:PutObject, s3:GetObject, s3:ListBucket).

**Check S3 Bucket Policy:**
Make sure the S3 bucket policy allows access to the IAM role.

**Use AWS CloudTrail:**
Check AWS CloudTrail to see the failing API calls and verify the identity under which the S3 access is attempted. This will help identify if the correct role is being used.

https://docs.aws.amazon.com/glue/latest/dg/what-is-glue.html

jub_atom · Answer

One more thing is that the SAP hana database is hosted on EC2 on another account. 
So, basically, My AWS glue job fetches data from SAP hana (hosted on EC2 on another account).
I am able to see the data on AWS glue cloudwatch output by doing dataframe.show()
However, while trying to write using glueContext.write_dynamic_frame.from_options, I am getting AccessDenied Error.
What could be the possible reasons and how to resolve this?

Access Denied on S3 from AWS Glue

Relevant content