Glue Error: error occurred while calling o228.pyWriteDynamicFrame. com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.s3.model.AmazonS3Exception: The specified bucket does not exist
I get this Error when run Glue ETL job:
Error Category: RESOURCE_NOT_FOUND_ERROR; An error occurred while calling o228.pyWriteDynamicFrame. com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.s3.model.AmazonS3Exception: The specified bucket does not exist (Service: Amazon S3; Status Code: 404; Error Code: NoSuchBucket; Request ID: NZ8QDBS69140YE35; S3 Extended Request ID: edcPcCDFP0u/wpMZODtwmEtqLav/gvf+ZrBLug6T1LEoxmhU5EzMIIS/cMhgvj4tLVyx8iLl46I=; Proxy: null), S3 Extended Request ID: edcPcCDFP0u/wpMZODtwmEtqLav/gvf+ZrBLug6T1LEoxmhU5EzMIIS/cMhgvj4tLVyx8iLl46I=
logs:
Glue Role:
VPC Endpoint for S3:
- Newest
- Most votes
- Most comments
Check the Bucket Name: Ensure that the S3 bucket name specified in your Glue script is correct. Even a small typo can cause this error.
Case Sensitivity: Remember that S3 bucket names are case-sensitive. Double-check the capitalization of the bucket name.
Go to the AWS S3 console and verify if the bucket exists.
Output Path: Double-check the output path specified in your Glue script for writing data. Ensure it is pointing to the correct bucket and folder.
Dynamic Frame Write: If you’re using pyWriteDynamicFrame, make sure the path specified in the method is correct:
Relevant content
AWS OFFICIALUpdated 3 months ago- AWS OFFICIALUpdated a month ago
In Redshift Target --> Performance and security --> S3 staging directory: s3://*******/temporary/ Not correct Bucket path I edit Bucket path and Create Folder temporary in this Bucket