Amazon Bedrock Knowledge Base with OpenSearch Serverless NextGen fails with Request Content Checksum Verification Failed

Hello.

I found a blog in Japanese that described the same problem you were experiencing.
According to the information provided, the issue appears to be caused by a content checksum mismatch in the SigV4 signature.
The blog does offer a workaround, but it involves querying OpenSearch manually instead of using the Retrieve API, which will likely increase the workload.
The blog post is based on information from June 1st, but I suspect this bug still hasn't been fixed as of July.
https://dev.classmethod.jp/articles/bedrock-kb-nextgen-retrieve-issue

Thank you for the response.

I verified the OpenSearch Serverless security configuration for the target collection.

Network policy:

• Public access is enabled for the collection endpoint.

Data access policy:

• The target collection has a data access policy that includes the Bedrock Knowledge Base service role used by this knowledge base as a principal.
• The policy grants the required collection-level and index-level permissions, including CreateIndex, DescribeIndex, ReadDocument, WriteDocument, and UpdateIndex.
• The index resource pattern is configured for the target collection, for example: index/<target-collection>/*.

Encryption policy:

• The collection uses the standard OpenSearch Serverless encryption configuration.

This same Bedrock Knowledge Base setup works when using an OpenSearch Serverless Classic collection. However, it fails only when using an OpenSearch Serverless NextGen collection with the following error:

[security_exception] Request Content Checksum Verification Failed (Service: BedrockAgentRuntime, Status Code: 400)

Based on the above, this does not appear to be a basic network policy or data access policy issue.

Can anyone from AWS confirm whether Amazon Bedrock Knowledge Bases officially supports OpenSearch Serverless NextGen collections as a vector store?

If NextGen collections are supported, are there any additional configuration requirements or known limitations compared with Classic collections?

Riku is correct. To add to that, the key is to query only the retrieval endpoint directly using opensearch-py. You should use AWSV4SignerAuth from opensearch-py and sign the request using the service name aoss.

ex.)

from opensearchpy import OpenSearch, RequestsHttpConnection, AWSV4SignerAuth

credentials = boto3.Session().get_credentials()
auth = AWSV4SignerAuth(credentials, "ap-northeast-1", "aoss")

client = OpenSearch(
    hosts=[{"host": "<collection-id>.aoss.ap-northeast-1.on.aws", "port": 443}],
    http_auth=auth,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection,
)

Thank you, this is very helpful.

The behavior described in the article matches my case exactly: the same Bedrock Knowledge Base setup works with an OpenSearch Serverless Classic collection, but Retrieve fails only with a NextGen collection due to a content checksum verification error.

I understand the workaround of querying OpenSearch Serverless directly with opensearch-py and AWSV4SignerAuth using the aoss service name.

However, in my case I would prefer to use the managed Bedrock Knowledge Bases Retrieve/RetrieveAndGenerate APIs rather than bypassing them.

So it seems that using a Classic collection is the safer option for now, unless AWS confirms that NextGen collections are officially supported and the checksum issue has been fixed.

It would still be helpful if AWS could confirm whether this is a known issue and whether OpenSearch Serverless NextGen collections are currently supported for Bedrock Knowledge Bases retrieval.

Thanks again for pointing me to the article and workaround.

Hello, I face the same issue : Bedrock Knowledge Base + Opensearch serverless NextGen, and when retrieving (using aws nodejs sdk), I get the Request Content Checksum Verification Failed error.

Bypassing bedrock and querying directly opensearch may be a solution, but I'd rather stay in the full bedrock environment.

Based on the available documentation about Amazon Bedrock Knowledge Bases with OpenSearch Serverless, I can provide guidance on the security and access configuration aspects of your setup.

For Amazon Bedrock to successfully access an OpenSearch Serverless collection as a vector store, you need to ensure proper configuration of several security policies:

Network Access Policy: The network access policy for your OpenSearch Serverless collection must explicitly allow Amazon Bedrock as a source service. This is done by adding "bedrock.amazonaws.com" to the SourceServices array in your network policy. This is particularly important for private collections.

Data Access Policy: You need a data access policy that grants the knowledge base service role appropriate permissions to the OpenSearch Serverless collection, including index-level permissions for reading and writing data.

Encryption Policy: If you're using AWS KMS encryption, the KMS key policy must grant the OpenSearch Serverless service (aoss.amazonaws.com) permissions to perform encryption operations (Encrypt, Decrypt, ReEncrypt, GenerateDataKey, and DescribeKey) with appropriate conditions restricting access to your AWS account.

Regarding your specific questions about OpenSearch Serverless NextGen collections and the checksum verification error: The documentation I have access to does not specifically address NextGen versus Classic collection compatibility or this particular error message. The "Request Content Checksum Verification Failed" error suggests a potential issue with how data is being transmitted or validated between services, which may not be resolvable through policy adjustments alone.

I recommend verifying that all three security policies (network, data access, and encryption) are correctly configured for your NextGen collection. If the policies are properly set and the issue persists, you may want to reach out through the AWS Support Center to investigate whether this is a compatibility or service-level issue specific to NextGen collections.
Sources
Set up security configurations for your knowledge base - Amazon Bedrock
AWS KMS key policies - AWS Prescriptive Guidance