- 最新
- 投票最多
- 评论最多
Thanks @Andy_P for all the suggestions.
After some days of monitoring NAT cost, I realized that the implementation of the S3 Gateway Endpoint it was actually working, the problem was that I thought that this change would be reflected right away in terms of costs, but I found out that this can take a bit more than 24 hours to be visible in AWS Cost Explorer.
From AWS docs: https://docs.aws.amazon.com/cost-management/latest/userguide/ce-exploring-data.html
All costs reflect your usage up to the previous day. For example, if today is December 2, the data includes your usage through December 1.
We already had AWS Flow Logs implemented in the VPC, so using the following query in Cloudwatch Logs Insight, I saw some reduction the first day, but I wasn't sure if it was a real reduction, or just casual less traffic:
# downloads in total
filter (dstAddr like '10.0.0.1' and not isIpv4InSubnet(srcAddr, '10.0.0.0/16')) | stats sum(bytes) as bytesTransferred
# uploads in total
filter (srcAddr like '10.0.0.1' and not isIpv4InSubnet(dstAddr, '10.0.0.0/16')) | stats sum(bytes) as bytesTransferred
So I needed to confirm that actually all inbound/outbound traffic between the subnets and S3 was going through the S3 Gateway Endpoint. After some research I found all AWS IP ranges here https://docs.aws.amazon.com/vpc/latest/userguide/aws-ip-ranges.html, then using this simple script to get only the S3 IP ranges:
import json
# Load the AWS IP ranges JSON file
with open('aws-ips.json') as file:
ip_ranges = json.load(file)
# Filter for S3 IPs in a specific region, e.g., us-east-1
s3_ips = [range["ip_prefix"] for range in ip_ranges["prefixes"]
if range["service"] == "S3" and range["region"] == "us-east-1"]
print(s3_ips)
I was able to write a more precise Logs Insight query to check for traffic between our NAT and S3, to check if there was still some traffic:
# downloads from s3
filter (
dstAddr like '10.0.0.1' and (
isIpv4InSubnet(srcAddr, '18.34.0.0/19')
or isIpv4InSubnet(srcAddr, '54.231.0.0/16')
or isIpv4InSubnet(srcAddr, '52.216.0.0/15')
or isIpv4InSubnet(srcAddr, '18.34.232.0/21')
or isIpv4InSubnet(srcAddr, '16.182.0.0/16')
or isIpv4InSubnet(srcAddr, '3.5.0.0/19')
or isIpv4InSubnet(srcAddr, '44.192.134.240/28')
or isIpv4InSubnet(srcAddr, '44.192.140.64/28')
)
) | stats sum(bytes) as bytesTransferred
# uploads to s3
filter (
srcAddr like '10.0.0.1' and (
isIpv4InSubnet(dstAddr, '18.34.0.0/19')
or isIpv4InSubnet(dstAddr, '54.231.0.0/16')
or isIpv4InSubnet(dstAddr, '52.216.0.0/15')
or isIpv4InSubnet(dstAddr, '18.34.232.0/21')
or isIpv4InSubnet(dstAddr, '16.182.0.0/16')
or isIpv4InSubnet(dstAddr, '3.5.0.0/19')
or isIpv4InSubnet(dstAddr, '44.192.134.240/28')
or isIpv4InSubnet(dstAddr, '44.192.140.64/28')
)
) | stats sum(bytes) as bytesTransferred
After running these queries, I confirmed there was no traffic, downloading nor uploading, between NAT and S3, right after the S3 Gateway Endpoint was deployed.
NOTE: A great tool is worth mentioning is AWS Reachability Analyzer, which I used to check connectivity between instance's ENIs in private subnets, and the S3 Gateway Endpoint.
Pls see this post to help troubleshoot the local S3 Gateway Endpoint.
For remote/inter-region S3 buckets have you considered S3 Multi-Region Access Points?
If Access Point restrictions are a blocker, consider the following:
- Establish private connectivity between local and remote regions if not already present (VPC Peering, Transit Gateway Peering)
- Create an S3 Interface Endpoint (not Gateway Endpoint) in the remote region
- If you want to resolve remote Region S3 to the global name (to avoid updating apps) you can use Private DNS support for Amazon S3 with AWS PrivateLink and Route53 Resolver Endpoints. Use an Outbound Endpoint in the local VPC to forward resolution of the remote S3 bucket to an Inbound Endpoint in the remote VPC. This will return the private S3 Interface Endpoint IPs for lookups made in the local VPC.
相关内容
- AWS 官方已更新 2 年前
- AWS 官方已更新 3 年前