My API Gateway is rate limiting and I want to prevent throttling. I want to select the best API Gateway Cache capacity for my workload.
Short description
Amazon API Gateway throttles requests to your API to prevent it from being overwhelmed by too many requests. Turn on API caching to reduce the number of calls made to your endpoint.
There are multiple API Gateway Cache sizes available. To select the appropriate cache size, run a load test on your API and then review the Amazon CloudWatch metrics.
Resolution
Turn on API Gateway caching
Turn on Amazon API Gateway caching for your API stage. The cache capacity depends on the size of your responses and workload.
Note: Cache capacity affects the CPU, memory, and network bandwidth of the cache instance. As a result, cache capacity can affect the performance of your cache.
After creating your cache, run a load test to determine if the cache size is high enough to prevent throttling.
Run a load test
Run a load test on your API. You can use AWS Distributed Load Testing to simulate the load test.
Run the load test for at least 10 minutes and mirror your production traffic. During the test, use the steps in the following section to monitor related CloudWatch metrics.
Monitor API metrics in CloudWatch
- Open the CloudWatch console.
- In the navigation pane, select Metrics.
- Choose the ApiGateway metric.
- Monitor Latency, 4XXError, 5XXError, CacheHitCount, and CacheMissCount metrics for the API that you are load testing against.
- If you see an increase in Latency, 4XXError, 5XXError or CacheMissCount with a CacheHitCount decrease, then resize your API Gateway cache to a larger capacity.
- If you see an increase in CacheHitCount and no corresponding increase in CacheMissCount, then resize your API Gateway cache to a smaller capacity.
- After any changes to your cache's capacity, run the load test again until there are no sudden increases or decreases.
Related information
Amazon API Gateway pricing
Turn on API caching to enhance responsiveness
Amazon API Gateway dimensions and metrics