How do I troubleshoot slow or inconsistent speeds when I download or upload data to Amazon S3 from an on-premises client?

Resolution

Decrease the distance to the client that makes the requests

It's a best practice to use Amazon S3 buckets that are as geographically as close as possible to your local client.

To test the effect of geographical distance between the client and the S3 bucket, complete the following steps:

1. Launch an Amazon Elastic Compute Cloud (Amazon EC2) instance in the same AWS Region as the bucket.
2. Launch another instance in a different Region.
3. Use both instances to upload and download the same file.
4. Compare the throughput between the two Regions.

To decrease the distance between the local client and the S3 bucket, take one of the following steps:

• Move your data into a bucket in a Region closer to the client. You can configure cross-Region Replication (CRR) to replicate the data in the source bucket into the destination bucket in the new Region.
• Migrate the client closer to the S3 bucket.
• Use S3 Multi-Region Access Points (MRAP) to provide a single global endpoint that automatically routes requests across regions to the nearest S3 bucket.
• Use the AWS Common Runtime (CRT), with SDKs that support the CRT such as AWS CLI v2, Boto3, or Java SDK 2.x. The CRT improves data transfer with parallel connections, intelligent retry logic, and multipart upload handling.

Check the local client's ISP

Review your network path for packet loss, a high number of hops, and other internet service provider (ISP)-related issues. Also review TCP retransmissions, SSL handshake failures, DNS resolution delays, or network proxies.

To determine potential packet loss and the latency of the packets that you send to the remote host, use tools such as mtr and traceroute. The tools can also identify whether a network hop is causing higher latency. For example, the following Linux traceroute command uses TCP port 80 to test connectivity to the us-west-2 (Oregon) Regional endpoint:

sudo traceroute -P TCP -p 80 s3.us-west-2.amazonaws.com

For Windows operating systems (OSs), run the following tracert command:

tracert -d -4 s3.amazonaws.com

Note: Many network devices don't respond to Internet Control Message Protocol (ICMP). For a test that's similar to Amazon S3 requests, run a TCP traceroute or mtr to the Regional endpoint of the bucket.

If the internet route from the client to Amazon S3 isn't optimal, then use Amazon S3 Transfer Acceleration for edge locations in Amazon CloudFront. Use the Amazon S3 Transfer Acceleration Speed Comparison tool to determine whether Transfer Acceleration can improve performance.

Note: If you turn on Transfer Acceleration, then additional data transfer charges might apply. Review Amazon S3 pricing for data transfers.

Check the local client's network

To confirm that your local client network is healthy, review internal packet inspections, antivirus scans, or network access management. Also, review how your client or application handles DNS resolution and caching. It's a best practice not to cache the DNS resolution for long periods of time.

To identify latency and performance issues from your on-premises client that has slow Amazon S3 sync transfers, use network diagnostic tests. It's a best practice to regularly analyze performance to detect potential issues before they affect transfer speeds.

Note: It's a best practice to use AWS PrivateLink for Amazon S3 for improved security and more reliable data transfers.

Check the client's resource usage

Your application can experience latency because of how the host that makes the requests handles the requests and responses. It's a best practice not to have resource contention within the host. For example, the resource contention might be with the CPU, memory, or network bandwidth.

To check the resource usage during data transfer, use tools such as Resource Monitor or the top command. For more information, see Resource Monitor on the Microsoft Learn website.

Note: Tool availability depends on your operating system.

Read or write operations to the client's storage device that's experiencing latency can affect the performance of the download from or upload to Amazon S3. To troubleshoot the IOPS of the storage device from the client side, use the FirstByteLatency Amazon CloudWatch metric to analyze Amazon S3 performance.

Note: AWS bills Amazon S3 request metrics at the same rate as custom CloudWatch metrics. For information about CloudWatch pricing, see Amazon CloudWatch pricing.

Check the request rate to Amazon S3

By default, S3 buckets can support thousands of requests per second per partitioned prefix. If a client gets HTTP 5xx error responses from Amazon S3, then the supported request rate per prefix is exceeded. To troubleshoot 5xx errors, see How do I troubleshoot an HTTP 500 or 503 error from Amazon S3?

To speed up transfer time with AWS globally-distributed CloudFront edge locations use Amazon S3 Transfer Acceleration.

To improve network configurations and enhance transfer performance, use AWS Direct Connect. For more information, see Improving performance on AWS and hybrid networks.

For design patterns that improve performance, see Performance design patterns for Amazon S3. You can use AWS DataSync, AWS Snowball, and AWS Transfer Family to migrate large datasets to Amazon S3. Or, you can use AWS SDKs and AWS Command Line Interface (AWS CLI) to create efficient parallel and batch transfer processes. To use multipart upload for large files, see How do I optimize performance when I use AWS CLI to upload large files to Amazon S3?

Review the Amazon S3 server access logs that determine the turn-around time

To review the Turn-Around Time metric, turn on Amazon S3 server access logging. For more information about the Turn-Around Time metric, see Turn-Around Time in Amazon S3 server access log format. Use the Turn-Around Time metric to determine the amount of time the operation takes. Then, you can evaluate other latencies outside Amazon S3 that might cause a slow download during upload.

If latency is higher than expected, it's a best practice to retry the request and not wait for the request to complete. For more information, see Performance guidelines for Amazon S3.

Check file transfer activities

Use the TotalRequestLatency metric to track the elapsed per-request time. This is the time from the first byte that the S3 bucket receives to the last byte that the local client sends to the bucket. To determine potential performance issues, check the GetRequests and PutRequests metrics and 4xx and 5xx status codes.

Note: AWS bills GetRequests and PutRequests metrics at the same rate as the Amazon CloudWatch custom metrics.

Related information

Amazon S3 request metrics in CloudWatch

How do I improve data transfer performance with the AWS CLI sync command for Amazon S3?