AWS Glue requires S3 VPC endpoint and AWS secrets manager although Glue is not in a VPC

Question

I have created a glue job which has a JDBC connection to Aurora serverless RDS which is attached to a vpc and a public subnet . I do not use S3 as a data source. However when I run the job the Glue job throws the error which says failed to find an S3 endpoint or NAT gaetway for the VPC - xxxx in subnet yyyyy .
On adding an s3 endpoint the glue job runs successfully .

Answer

Hi, there,

I understand that when you are getting “failed to find an S3 endpoint or NAT gaetway for the VPC - xxxx in subnet yyyyy” and after adding the s3 endpoint it works.

As per the below article, either adding the S3 VPC gateway endpoint or a route to a NAT gateway in the subnet's route table, should resolve this error. For more details, please refer the below article:

[+] https://repost.aws/knowledge-center/glue-s3-endpoint-validation-failed

Answer

The reason that this is an issue is because Glue stores its scripts and a temporary files in an S3 bucket.

Either an S3 gateway or a NAT gateway will provide the appropriate access to S3.

AWS Glue requires S3 VPC endpoint and AWS secrets manager although Glue is not in a VPC

Relevant content