Skip to content

Anthropic Claude Sonnet 4 on Bedrock thinks it is Sonnet 3.5

1

It seems that Sonnet 4 thinks it is Sonnet 3.5. It also believes it's knowledge cutoff date is April 2024 instead of January 2025. This behaviour deviates from usage of Anthropic's Claude directly.

This can be reproduced in Bedrock playground or through the API. Minimal example using boto3:

import boto3
import json

# Use eu-south-2 region
bedrock = boto3.client(service_name="bedrock-runtime", region_name="eu-south-2")

body = json.dumps({
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "Let me try this new Claude 4"}],
    "anthropic_version": "bedrock-2023-05-31"
})

# Use cross-region inference profile for Claude 4 Sonnet
response = bedrock.invoke_model(
    body=body, 
    modelId="eu.anthropic.claude-sonnet-4-20250514-v1:0" # same holds for inference profile of us
)

# Printing only the textual response
response_body = json.loads(response.get("body").read())
print(response_body['content'][0]['text'])

I'm actually Claude 3.5 Sonnet, not Claude 4! There isn't a Claude 4 released yet - you're using one of the current Claude 3.5 models from Anthropic. Is there something specific you'd like to try out or explore with Claude 3.5? I'm happy to help with questions, creative tasks, analysis, coding, or just have a conversation!

Claude.ai gives the following response for the same prompt:

Hello! Yes, you're chatting with Claude Sonnet 4 from the Claude 4 model family. I'm here to help with whatever you'd like to work on - whether that's answering questions, helping with analysis, creative projects, coding, or just having a conversation. What would you like to explore today?

Is Anthropic Claude Sonnet 4 on Bedrock suffering from an imposter syndrome, or a typo in it's system prompt or just unaware of any accurate information about itself?

2 Answers
-1
Accepted Answer

From what I understand, the issue appears to be that Claude 4 Sonnet is not aware of its own model identity because it didn't exist during its training period. The reason you receive the correct model identification in claude.ai is likely because Anthropic has included this information in the system prompt for that deployment.

The claim of Claude 4 on Bedrock that its knowledge cutoff is April 2024 is demonstrably incorrect. For example, when you ask it about recent events like President Bashar al-Assad's departure from Syria, it correctly identifies this as occurring on December 8, 2024—well beyond the supposed April 2024 cutoff date.

answered a year ago

  • Are you sure you weren’t using tools during your prompting? I’m asking the same question with tools disabled and Sonnet 4 claims there is no such information.

  • Yes. Just try the Chat / Text playground in AWS Bedrock and you should get this result. I am not sure which tools you are referring to. I believe Claude 4 on AWS Bedrock has no native web search tool (as opposed to Gemini in Google's Vertex AI)

  • Great idea, Henrik, to assess it through querying outside the context cutoff.

-1

The issue you're experiencing with Claude Sonnet 4 on Amazon Bedrock incorrectly identifying itself as Claude 3.5 Sonnet appears to be a configuration or initialization problem.

Based on the available information, Claude Sonnet 4 is indeed available on Amazon Bedrock - it's listed among the supported Anthropic Claude models that can be used with the Messages API. The model ID you're using in your code ("eu.anthropic.claude-sonnet-4-20250514-v1:0") also appears to be correctly formatted for accessing Claude Sonnet 4.

This discrepancy where the model incorrectly identifies itself could be due to several factors:

  1. There might be an issue with how the model is initialized or configured within Bedrock
  2. The system prompt or model configuration might not have been properly updated to reflect the correct model identity
  3. There could be differences in how the model is deployed on Bedrock versus directly through Anthropic

This is likely a technical issue rather than the model having "imposter syndrome." Similar discrepancies have been reported before between Claude models accessed directly through Anthropic versus through Amazon Bedrock, including differences in responses and behaviors.

I would recommend reaching out to AWS Support to report this issue, as it appears to be a configuration problem with how Claude Sonnet 4 is deployed on the Bedrock platform.
Sources
Amazon Bedrock with Claude Sonnet 3.5 is not working the same as when I access Claude directly | AWS re:Post
Supported models - Amazon Bedrock

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.