increasing Bedrock quotas without provisioning throughput

0

Hi. Is there a way of increasing AWS Bedrock quotas without purchasing Provisioned Throughput?

For example if you want to create an app that summarizes books without chaining prompts you are limited to Anthropic's models. In production, if you have some clients it will throttle as you would reach the 200,000 tokens per minute easily (https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html). Is there an alternative to increase that limit without incurring in the huge costs of provisioned output? For example provisioning 1 model unit for Claude 2.1 costs around $45,990.00 :/. I guess the provisioned throughput works like EC2 instances, you are billed by the provisioned capacity not by the use, so although it is not used you would be billed that? It seems like there is not an intermediate option.

Thanks!

1回答
3
承認された回答

Hi Victor,

Currently there is no way to increase AWS Bedrock quotas without purchasing Provisioned Throughput. This has been asked commonly, and there are no workarounds besides Provisioned Throughput as of now.

I would recommend reaching out to your Account Team to have them push forward this request toward the service team.

Another workaround is that you could use a multi account strategy, as the token limit per minute is an account level limit. This is certainly not the best developer experience but might be a solution for now.

AWS
autrin
回答済み 2ヶ月前
profile picture
エキスパート
レビュー済み 2ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ