- Newest
- Most votes
- Most comments
The issue was within how boto3 handles different aws regions. This may be unique to usage on AWS GovCloud. Originally I did not have a region configured for S3, but according to the docs an optional environment variable named AWS_S3_REGION_NAME can be set.
AWS_S3_REGION_NAME (optional: default is None) Name of the AWS S3 region to use (eg. eu-west-1)
I reached this conclusion thanks to a stackoverflow answer I was using to try to manually connect to s3 via boto3. I noticed that they included an argument for region_name when creating the session, which alerted me to make sure I had appropriately set the region in my app.settings and environment variables.
If anyone has some background on why this needs to be set for GovCloud functionality but apparently not for commercial, I would be interested to know.
More details can be found in my related question.
What I'm understanding here is that you're concerned that use of GoDaddy DNS has affected your application's access to S3. It shouldn't because all you need for your application to reach the S3 service is either outbound internet access from the VPC or an S3 VPC endpoint - it would be best to make sure this is working ok. Unless you're deploying your own DNS Servers in the VPC to handle DNS resolution within the VPC (not the internet as GoDaddy is doing) then you're using the Amazon DNS servers for resolving an S3 domain name when you access S3.
An S3 VPC endpoint can be a Gateway or Interface one. You mentioned a PHZ - if you did use an Interface endpoint by default it sets up an AWS-Managed PHZ to override the S3 domain name resolution, resolving it to the endpoint's private IP address instead, but you can choose to self-manage this instead (not normally required unless you're sharing VPC endpoints across accounts - see https://www.linkedin.com/pulse/how-share-interface-vpc-endpoints-across-aws-accounts-steve-kinsman). But usually unless you need to access S3 from on-prem you would just use a Gateway endpoint as there's no cost.
Relevant content
- Accepted Answerasked a year ago
- Accepted Answerasked 2 years ago
AWS OFFICIALUpdated 2 years ago- AWS OFFICIALUpdated 3 months ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 3 months ago
Thank you for the information on how Amazon handles intra-VPC DNS resolution. I already have a VPC Gateway endpoint set up for S3 with a security group attached allowing http:80 and https:443 from another security group that is attached to my EC2-ECS cluster. Despite this I still cannot connect to S3 from the container; however, I can connect to the S3 bucket if I ssh into the EC2 instance running the containers and use the aws cli (tested by running `aws s3 ls s://BUCKET_NAME.s3.amazonaws.com').