Claude on Bedrock giving "Input is too long for requested model" for ~10k token inputs (edit: broken in eu-central-1, working in other regions)

Question

When using Claude on Bedrock (v1.2 or v2) I get the following error when the context approaches ~10500 tokens in length:

> An error occurred (ValidationException) when calling the InvokeModel operation: Input is too long for requested model.

If I shorten my context to the point where it just works using the Python client, the request succeeds with 10432 prompt tokens and 450 completion tokens. If I add one more character to the prompt it fails with the above error. The same error happens via both the API (using Python client) and in the playground, although the playground doesn't give an error it just fails to load a response.

Using Anthropic's API Claude doesn't error until the context is nearly 100k in length. Is there an unknown limit in Bedrock's implementation?

[Here is a prompt](https://www.dropbox.com/scl/fi/tixedk1zf9w2i8485yrq6/test_bedrock_long_prompt.txt?rlkey=486r65hvnadhutfdglnzql1q5&dl=0) that is significantly less than 100k tokens (it's 88700 characters) and should only require a few hundred tokens for the completion.

Answer

Hi,

Your long prompt goes fine with Anthropic Claude V2 on Bedrock.

This is the answer that I got with your prompt:

```
Stops for applause: 20

Rating: Positive
Reason 1: The speech promotes optimism about America's future and resilience in the face of challenges. References 
America's determination and spirit which lives on in its people.
Reason 2: Calls for bipartisan cooperation to solve problems and emphasizes common ground over differences. 
References need to "find a way to come together and finish the job for the American people."  
Reason 3: Highlights progress made in the past year on the economy, jobs, financial reform, clean energy and 
innovation, education, veterans affairs, and national security. Gives many specific policy achievements and plans.

Theme 1: Economic recovery and job creation
Description: Discusses steps taken to rescue the economy, stabilize banks, cut taxes, pass Recovery Act, and create/save jobs. 
Calls for new jobs bill and lays out proposals to support small businesses and infrastructure projects.

Theme 2: Bipartisan governance and reform of political system  
Description: Emphasizes need to overcome partisan divides and govern together. Proposes reforms to lobbying, disclosure,
 and earmarks. Calls for fiscal responsibility and bipartisan fiscal commission.

Theme 3: Investing in America's future through education, energy, innovation  
Description: Outlines plans to reform education, make college more affordable, invest in clean energy/technology, 
and continue basic science research. Presented as necessary to address long-term challenges and compete globally.
```

Also, please, check if we align on query parameters (right side of screenshot)

![Enter image description here](/media/postImages/original/IMjeBZnsn-SV-Mbdhy6DVigQ)

Best,

Didier

Answer

So i re-tested this last night in eu-central-1 and managed to send about 70k tokens through and it worked so think someone might have fixed it. Let me know if someone else is able to double-check this.

Answer

Yeah same for me today. Still no joy - tested exactly same config except switching to diff region and worked fine. Definitely issue specific to Frankfurt which is frustrating as only Bedrock instance within EEA atm.

Answer

We're seeing this issue in US-East-1 using the Claude v2 (100k). It works when we set our max_tokens to 61,551 tokens but the moment we it to 61,552 our request fails (regardless of the size of our text sent in our request).

Can anyone else verify this in us-east-1 as well by upping their max_tokens value?

Claude on Bedrock giving "Input is too long for requested model" for ~10k token inputs (edit: broken in eu-central-1, working in other regions)

Relevant content