Cross-account S3 Bucket access in AWS Glue Crawler
Hi Team,
I have setup Cross-account IAM roles to access a s3 bucket in an extremal AWS Account following this https://repost.aws/knowledge-center/cross-account-access-s3. Not following a bucket policy here. I am assigning this IAM role to AWSGlueServiceRole along with other roles for Glue. However, I am still getting an AccessDenied exception on S3. Am I missing any permission ? Attached is the existing Policies attached to AWSGlueServiceRole
- Newest
- Most votes
- Most comments
Hi,
For glue crawler, you should not be following the same way as mentioned in that knowledge center article, assumerole may not work for glue crawler case.
There are few things you need to keep in mind for cross account s3 access:
- Crawler role in Account A should have access to Account B s3 bucket(Get*, List*)
- Account B s3 bucket must allow required permissions(Get, List etc) to account A crawler role in it's bucket policy.
- Account B s3 bucket must not be using SSE-KMS(aws/s3) key, if bucket is encrypted with aws/s3 AWS Managed KMS key then cross account s3 access won't work
- If Account B s3 bucket is SSE-KMS CMK(custom key) encrypted then, KMS key policy in Account B must allow Account A glue crawler role.
Hope this helps.
Comment here if you have additional questions, happy to help.
Abhishek
Relevant content
AWS OFFICIALUpdated a year ago- AWS OFFICIALUpdated 5 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 months ago
Thanks Abhishek. For my use case, the Customer's s3 bucket is not enabled with any SSE-KMS(aws/s3) key nor any Customer managed key. Do you think bucket policy is the only option ? Customer's s3 bucket hosts all the AWS Billing Cost and Usage (CUR) data and I am afraid, they can alter the bucket policy for that bucket. This is because AWS suggests not to alter the bucket policy holding the billing data. Hence I setup a Cross Account data access using assume role. Kindly suggest. Thanks!
For cross account s3 bucket access, target account bucket policy must allow source account role. From your description it seems that you are trying to achieve role chaining where glue crawler can assume target account role, but to best of my knowledge you may not be able to do that. You can do role chaining with proper permissions setup as mentioned in the article which you highlighted but for that you need to make an sts:assumerole api call explicitly, in case of glue crawler that would not be possible.