How do I troubleshoot an HTTP 500 or 503 error from Amazon S3?

Short description

When you make a request to Amazon S3, you might receive an error message similar to the following examples:

• "AmazonS3Exception: Internal Error (Service: Amazon S3; Status Code: 500; Error Code: 500 Internal Error; Request ID: A4DBBEXAMPLE2C4D)"
• "AmazonS3Exception: Slow Down (Service: Amazon S3; Status Code: 503; Error Code: 503 Slow Down; Request ID: A4DBBEXAMPLE2C4D)"

The "500 Internal Error" error message occurs when Amazon S3 can't manage the request at that time. The "503 Slow Down" error message typically occurs when your S3 bucket receives a high number of requests. You can send 3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second for each partitioned Amazon S3 prefix. However, Amazon S3 might sent a "503 Slow Down" response if your requests exceed the bandwidth that's available to you to copy across AWS Regions.

To resolve or avoid 5xx status errors, take the following actions:

• For the application that makes the requests, use a retry mechanism.
• Configure your application to gradually increase request rates.
• Distribute objects across multiple prefixes.
• Monitor the number of 5xx error responses.

Note: When you create a prefix, Amazon S3 doesn't automatically assign additional resources for the supported request rate. Amazon S3 scales based on request patterns. As the request rate increases, Amazon S3 dynamically optimizes for the new request rate.

Resolution

Use a retry mechanism

Because of the distributed nature of Amazon S3, you can retry requests that return 500 or 503 errors. It's a best practice to build retry logic into applications that make requests to Amazon S3. AWS SDKs have a built-in retry mechanism.

Note: In some scenarios, rapid concurrent requests to the same key can result in a 503 response. In these scenarios, it's a best practice to retry failed requests.

Configure your application to gradually increase request rates

A high request rate or a sudden increase in the request rate for an object can result in the "503 Slow Down" error message. Configure your application to maintain the request rate and implement a retry with exponential backoff. Exponential backoff implements increasingly longer wait times between retries for consecutive error responses. This configuration allows Amazon S3 time to monitor the request patterns and scale in the backend to manage the request rate.

First, configure your application to start with a lower transactions per second request rate. Then, exponentially increase the application's request rate. Amazon S3 automatically scales to manage a higher request rate.

Distribute objects across multiple prefixes

Request rates apply for each prefix in an Amazon S3 bucket. To set up your bucket to manage higher request rates, distribute objects across multiple prefixes. For example, if you use a bucket to store images and videos, then distribute the files into the following two prefixes:

• mybucket/images
• mybucket/videos

If the request rate on the prefixes gradually increases, then Amazon S3 scales up to manage requests for each prefix separately. As a result, the bucket can manage double the request rate.

Monitor the number of 5xx status error responses

To monitor the number of 5xx status error responses that you receive, use one of the following options:

• Activate Amazon CloudWatch metrics to check the 5xx status response metric.
• Activate Amazon S3 server access logging to capture all requests, and then filter and review requests that receive a 5xx error response. Use Amazon Athena to parse the logs.

Troubleshoot further issues

When you use the Expedited option to retrieve archived objects, you might receive one of the following error messages:

• "GlacierExpeditedRetrievalNotAvailable"
• "Glacier expedited retrievals are currently not available, please try again later"

These issues occur if there's insufficient capacity to process the Expedited RestoreObject request. During a period of sustained high demand, Amazon S3 might deny Expedited retrieval requests and return a 503 error message. Use provisioned capacity units to make sure that there's Expedited retrieval capacity available on demand. You can perform at least three expedited retrievals every 5 minutes for each unit. Each unit provides up to 150 MBps of retrieval throughput. You can also use Standard or Bulk retrieval options.

You can retry the retrieval. However, you might still encounter issues. Except in cases of extreme demand, Expedited retrievals are possible without provisioned capacity. Because of the constant change and high demand for Expedited retrieval availability from non-provisioned capacity, there's no Expedited retrieval SLA.

If you continue to receive a high amount of 5xx status error rates, then contact AWS Support. Include multiple Amazon S3 request ID pairs for requests that fail with a 5xx status error code.

Related information

Using S3 Storage Lens metrics to improve performance

Monitoring metrics with Amazon CloudWatch