Bedrock: "Too many requests, please wait before trying again."
I've been creating a POC using bedrock and I've started receiving the error message "Too many requests, please wait before trying again.". This is for programatic and console (i.e. bedrock playgrounds) operations.
I understand that there is a quota limit however, I'm well under the limit for requests per min and tokens.
It seems to be linked to only the US regions (all models), Sydney is working fine but doesn't have the model that I require. Further, the service status notes that bedrock is available in the US regions.
What could be causing this, and are there any solutions?
I've been able to invoke a model 3 times in the last week! The below is my cloudwatch metrics.
As this issue is occurring during the development of a POC it's not a great sign for a production application. If it persists I'll have to use an external provider.
- Newest
- Most votes
- Most comments
I have now confirmed the issue. It appears some bug where the models we had previous been grant access were somehow set to a quota of 0. After requesting access to 2 new models. While using them, I get no errors. THese models have a non-zero quota set.
Jamba Large 1.5 and Jamba 1.5 are the only models I am able to use without gets the error. I have access to all of the other models here in Bedrock Model Access, however, since my Account Level Quota has been set to 0 I, as well as many of you I suspect, are getting the 'Too Many Requests' error. Very upsetting that this happened and there has been no response from AWS.
yeah, mine is the same as this. It's such a pain as there is no way of requesting an increase for the on-demand models
Yes, I am experiencing the same and noticed that quota is set to 0 :-( Not sure if there is a bug or another issue that is causing this. If anyone able to figure out, please share the feedback.
Same problem here. Really frustrating and does not help with using AWS to build an AI PoC. Support request in progress to reset quotas to >0
Following up on this issue.
- This was an unusual issue and not related to the number of user invokations nor token limits
- AWS had updated on-demand quotas to 0 for some regions and models
- Myself and others needed to open a support case, where aws support were able to update the quotas for the effected accounts
- If you don't have "Technical support", just raise it through the general/billing one
- It takes ~48 hours to resolve (edit: via the billing support)
AWS has done something at the backend that affects all accounts for all customers, I am unsure if I missed any communications. Last week I noticed that my AWS Lambda concurrency was 10 instead of a normal 1000, due to being affected by this change, I logged a quota increase request. It turns out AWS dropped this value for all new accounts to help customers avoid charges for functions that run away.
I suspect AWS did the same on Amazon Bedrock service, and possibly many other services to help customers minimize cost with unintended API calls to Amazon Bedrock, etc...
"ThrottlingException: An error occurred (ThrottlingException) when calling the RetrieveAndGenerate operation (reached max retries: 4): Your request rate is too high. Reduce the frequency of requests."
These errors I'm getting when calling Bedrock from the SDK are not clear on what quota is being exceeded. I searched the service quotas page for Amazon Bedrock with no luck.
It's not us, there is something that changed for all customers, we just need to find out what is it, and then request a reset back to what it was.
Yes it appears that this is what happened. I can't find any new information from Amazon Bedrock detailing any changes to the rate limits. It may be worth us making an Amazon Support Tickets to hopefully bring this issue to there attention or find a resolution.
Good day,
We started experiencing this same issue today out of nowhere. Been running great for months. Now even after being idle for hours (0 requests made) even a single request using the Playground in Bedrock yields the same error. "Too many requests". Even a fresh new account created gives the same error. Trying on and off, since 15 hours ago, not a single request from within Bedrock's Playground, nor any of our Websites and Apps have succeeded.
After trying a bunch of things, I noticed that some regions work, and others not. So for now, this seems like an outage to us (We are in South Africa) for certain regions like US-East and West. London works though. So we have no choice but to move our KB's there to at least get our services up and running again.
similar here - we're in us east and began running into this about 8 hours ago. our usage is up but not much different so the assumption is this is more global. unfortunately we can't event submit a support case to aws because of a glitch in their system where there's no option to select a severity for 'bedrock' service
Managed to get a ticket escalated to the Bedrock team at Amazon. They do not seem to be aware of a general issue. So wondering why we have been affected by this specifically. Anyways, the Quotas seem to be the issue, and they will have a look and get back to me. Will update here when/if something changes.
i'm also getting the same issue. all my services are in us east-1. nothing is working except the amazon titan text premier and some ai21 models.
AWS Support has assisted us, Quotas are restored again and services singing. Reach out to them to assist you if still having issues.
It appears that perhaps the quotas for invoking several models have been set to 0 ifor us east 1 instead of the default AWS values for my account.
https://us-east-1.console.aws.amazon.com/servicequotas/home/services/bedrock/quotas
I've opened a support case with my personal account (no expensive support plans) and they solved it in 48 hours.
I had a similar experience to what is being discussed here. I started noticing the getting "Too many requests, please wait before trying again" errors on Oct 24, and it seems the problem started sometime after Oct 17 . I was getting the error response from Bedrock "invokeModel" API requests, and also from the Bedrock Playground UI.
To resolve the issue, I opened a Basic Support plan case with AWS Support using the following form: https://support.console.aws.amazon.com/support/home#/case/create
- Select "Account and Billing"
- Service: General Info and Getting Started,
- Category: Using AWS & Services
- Severity: General Question
The AWS Account and Billing Support team were very helpful and connected with the back-end service team to troubleshoot and resolve the issue. The issue was resolved within about 48 hours; I did not receive any details on the root cause (which is fine, this is a Basic Support plan after all.)
Thanks @tommi, your answer works for me. I have many accounts with the same problem, and AWS is working to solve that quota problem.
Also getting this for the first time today. Following. Did tracking tokens work? I am getting this message even when I try an identical flow in the Bedrock Prompt Flow Builder:
Too many requests, please wait before trying again. You have sent too many requests. Wait before trying again. (Service: BedrockRuntime, Status Code: 429, Request ID: x
No clue why it would be too many or where I can even check what is going on.
I am also suddenly getting this error for my SaaS tool within the past 12 or so hours. I have no dramatic changes in usage and have not previous encountered this error. My SaaS tool has been operating at it's current level for months now so this error is causing a severe degradation of the service.
Both the Converse API and the older Invoke Model methods, yield the same error.
I opened a case with Billing support and they resolved my case. I am able to run agents now again. They looked at our account and changed the quotas from 0 to AWS default.
I told them in my submission: I am getting "Too many requests" errors in Bedrock testing agent models but my utilization is very low. I found that in Quotas, our account quota is 0 (for example, "On-demand InvokeModel requests per minute for Anthropic Claude 3 Sonnet", but the AWS default quota value is anywhere from 50 to 1000. For the quotas I need to request an increase, they are not selectable, so I am unable to request increases from the quota page. I don't know if this is a billing matter because our account is new.
Same for me. The errors "Too many requests, please wait before trying again. You have sent too many requests. Wait before trying again.. Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException'" started appearing on 2024-10-21T05:03.
And there are "Not adjustable" quotas with zero values for the model I use and almost all others:
There are no issues related to Bedrock displayed in AWS Health.
Can I report AWS bug on a Basic Support plan? When I try to contact support it shows that I need at least "Developer" (from $29.00 / month) support plan to contact them about technical issues.
AWS support restored quotas. I created a free support request (Account/General question).
thanks for sharing, that is good to know. did they say what caused the prob? can you please share how long it took them to respond back and fix it?
I developed my client code this weekend and hit this, anybody working again?
raise a support ticket and aws will restore the defaults. Go through the technical support if you have it, otherwise using the billing one. Mine has just been restored
Also seeing the same issue, filed a ticket to have the service quotas restored, support said they expected it be be done Monday, 2024-10-28.
This is a major problem though, because you need to have AWS business support which runs $100/month minimum in order to file a tech support ticket. I burned a 2 free months of support promotion in order to be able to file, but obviously this isn't an ideal solution.
I have same issue. Either claude or llama can be used, all quota are set to 0 and contact support just tell me "wait", they are "investigating"
Has anyone got there problem solved? I cut a ticket last saturday but still hasn't got the problem solved.
Hi,
This throttling exception is normal if you go over quota: you have to manage your calls to Bedock to avoid it. In my application, we do that with a standard exponential backoff mechanism: https://en.wikipedia.org/wiki/Exponential_backoff
The first thing to manage are the number of calls per minute.
But, in many applications, you reach the quota limit not due to requests but to LLM token consumption. So, you have to manage this as well to avoid throttling.
In my app, we reach throttling because of the large contexts that we supply to the LLM: so, input tokens are the reason why we reach the throttling limit (not the number of requests).
Bedrock returns the token consumption in the http response headers: you have to accumulate and manage them to avoid the reach of throttling.
To get them:
input_token_count = http_headers.get('x-amzn-bedrock-input-token-count')
output_token_count = http_headers.get('x-amzn-bedrock-output-token-count')
Best,
Didier
Thanks for the quick response and advice Didier!
Do you know when should I expect the quotas/limits to reset (daily, monthly, etc.)?
I've previously reached the token limits when I was repeatedly sending invocations, I gave it a short period of inactivity and the service resumed as normal. This time however, the limits haven't reset for the region for a few days now.
Could it be that maxing out the quotas at the start of the month has caused this?
This does not appear to be the case for the error in this post. Something happened on AWS Bedrock's backend that set our Invocations per minute Quota to 0. This means no amount of backoff will work. You can check your quotas here: https://us-east-1.console.aws.amazon.com/servicequotas/home/services/bedrock/quotas
Relevant content
- asked 3 months ago
AWS OFFICIALUpdated 3 months ago- AWS OFFICIALUpdated 2 months ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 2 months ago
I am also seeing this in us-east-1 and us-west-2 across llama3, llama3.1 and Amazon titan. I haven't hit my quota as well. I don't see any outage in health checks. Can it happen to all models at the same time if it is a shared pool?
I'm also getting the same issue.
I just realized I used my Builder ID. Switched to my internal AWS profile here on re:Post now. To double-check, click on my name above this comment to confirm I'm legit :-)
Same issue here, we only used a small amount of Bedrock services this last month as we are still ramping up, well under the limit but we are getting the too many requests for all requests. It seems to be most bedrock models are affected too. I have reached out via email as you mentioned, thanks much for any assist.
Same issue here. I tried to access Bedrock via AWS SDK for Javascript V3, and I have the same error "Too many requests, please wait before trying again." (status code: 429). Always with the same error. In this case, I used the Anthropic's model, Claude v2. Next, I tried to access the playground of the Bedrock in AWS management console, but I had the same error. In this case, I used not only Claude v2 but also Titan, but the same error returned.