increasing Bedrock quotas without provisioning throughput

0

Hi. Is there a way of increasing AWS Bedrock quotas without purchasing Provisioned Throughput?

For example if you want to create an app that summarizes books without chaining prompts you are limited to Anthropic's models. In production, if you have some clients it will throttle as you would reach the 200,000 tokens per minute easily (https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html). Is there an alternative to increase that limit without incurring in the huge costs of provisioned output? For example provisioning 1 model unit for Claude 2.1 costs around $45,990.00 :/. I guess the provisioned throughput works like EC2 instances, you are billed by the provisioned capacity not by the use, so although it is not used you would be billed that? It seems like there is not an intermediate option.

Thanks!

1개 답변
3
수락된 답변

Hi Victor,

Currently there is no way to increase AWS Bedrock quotas without purchasing Provisioned Throughput. This has been asked commonly, and there are no workarounds besides Provisioned Throughput as of now.

I would recommend reaching out to your Account Team to have them push forward this request toward the service team.

Another workaround is that you could use a multi account strategy, as the token limit per minute is an account level limit. This is certainly not the best developer experience but might be a solution for now.

AWS
autrin
답변함 2달 전
profile picture
전문가
검토됨 2달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠