Amazon Bedrock Claude 3 Opus - Throttling Rate Limit - How long until reset?

Question

Recently I received an error while using Amazon Bedrock Anthropic Claude 3 Opus:

ThrottlingException: An error occurred (ThrottlingException) when calling the InvokeModelWithResponseStream operation (reached max retries: 4): Too many requests, please wait before trying again.

An hour later, I retried and still received this error. 
**How long until this is reset?**

If I were to use "InvokeModelWithResponse" but not with "Stream" (Streaming response) would I still get this rate limit? Can I increase the number of retries or is this a hard limit?
I'm only sending one message to it every few minutes.. surely there is something wrong? Or are the rate limits really set this low?

Thanks in advance!

Accepted Answer

Hello!

I send you the below rePost question previously answered for similar error regarding ThrottlingException, is for Claude v2.1 but it applies in the same way to Claude v3:

* https://repost.aws/questions/QU11DRlMZfRDy0ngHxpO1VCw/throttlingexceptions-while-using-on-demand-bedrock-runtime-for-invoking-claude-v2-1

Here one of the possible solutions to this problem, provisioned throughput provides reserved capacity for your account specifically:
* https://docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html

Answer

Hi,

As you are using an **on-demand model** that makes use of a shared capacity pool, during periods of high demand across the service your account may be throttled below their expected rates.

The internal Bedrock team is working on expand capacity and address this issue, but currently we don't have an ETA in place for capacity increases.

Meanwhile you can try changing the model or the region if your use case allows it, or purchasing the provisioned throughput (see the upper link) to have allocated capacity to your account.

Answer

I'm also getting the ThrottlingException when trying to use Claude 3 Opus, regardless of when and how many times I try. Do I need to change some hidden configuration? Is this a known problem?
Thank you.

Amazon Bedrock Claude 3 Opus - Throttling Rate Limit - How long until reset?

Relevant content