AWS Bedrock - Anthropic Claude V2 Max tokens


When invoking the model ("modelId": "anthropic.claude-v2") using boto3 version 1.28.59 with "max_tokens_to_sample" higher than 8191. I get an error: botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: 60000 is not less or equal to 8191, please reformat your input and try again.

One one the reasons I am looking into Claude 2 is its 100K tokens limit. Do I need to request getting 100K?

Thank you in advance! Max

Max M
asked 8 months ago4527 views
2 Answers
Accepted Answer

My bad. The documentation ( clearly states that the longest output is 8K. The context size can be 100K, but the length of the response cannot exceed 8K. My use case was to decompose a document into Q&As, so my output is about the same size as the input and I will need to chunk documents into smaller pieces.


Max M
answered 7 months ago
profile picture
reviewed 2 months ago

The input/output sizes can change and contains the latest info as of this writing :)

answered 3 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions