Skip to content

OpenSearch Serverless NextGen: Bedrock Knowledge Base creation fails with "Recreate your vector index with FAISS" even with AWS's own documented examp

1

I'm trying to use an OpenSearch Serverless NextGen collection group (for scale-to-zero cost savings) as the vector store behind an Amazon Bedrock Knowledge Base, in eu-west-2.

Configuration:

  • Collection group: generation NEXTGEN, standby_replicas ENABLED, capacity_limits with 0 min OCU
  • Collection: type VECTORSEARCH, linked to the group
  • Region: eu-west-2

Issue:
Creating the vector index succeeds, but bedrock-agent:CreateKnowledgeBase always fails with:

ValidationException: The knowledge base storage configuration provided is invalid...
The OpenSearch Serverless engine type associated with your vector index is invalid.
Recreate your vector index with FAISS as the engine type and then retry your request.

The catch: NextGen collections reject an explicit "engine" field in the index mapping entirely:

  • 400: Field parameter 'engine' is not supported *

I followed AWS's own launch blog walkthrough exactly (https://aws.amazon.com/blogs/big-data/the-next-generation-of-amazon-opensearch-serverless-built-from-the-ground-up-for-agents/), which
shows creating an index via awscurl with an explicit "engine": "faiss" field as a working example. Reproducing that exact request with awscurl against my own collection gets the same 400
rejection.

To rule out an account-specific fluke, I also created a brand-new, isolated NextGen collection group + collection from scratch and got the identical error within minutes of creation.

I've also seen other users get past this exact step successfully with NextGen — for example:
https://repost.aws/questions/QUvVECr8qvTEC52iuxfGJ7TA/
https://dev.classmethod.jp/en/articles/bedrock-kb-nextgen-retrieve-issue/

2 Answers
0

A DEV Community article "I tested the new Amazon OpenSearch Service Agent Toolkit skill. Here's what it actually does. - DEV Community" pinpointed this exact issue.

Engine selection for NextGen. The skill recommended "engine": "faiss" which fails on NextGen with illegal_argument_exception. NextGen manages vector acceleration internally, you just specify dimension and space_type.

In other words, the behavior you observed—where NextGen collections completely reject the engine field (returning "400: Field parameter 'engine' is not supported")—is actually the correct specification for the NextGen architecture. NextGen automatically manages the vector engine (an implementation equivalent to FAISS) internally; the design does not require users to explicitly specify the engine (and doing so actually causes an error).

NextGen specification: The engine field is unnecessary and causes an error; it is managed internally.

Bedrock CreateKnowledgeBase validation logic: It expects the index to have explicit metadata (engine: faiss) attached; if this is missing (or undetectable), it returns an "Invalid engine type" error.

Recommended Action

  • I strongly recommend opening an AWS Support case. The validation steps you performed—following the official blog instructions exactly, verifying directly via awscurl, and confirming reproducibility with a brand-new collection group—are highly compelling and provide sufficient evidence to argue that there is an incompatibility between the "NextGen + Bedrock Knowledge Base" combination itself. Citing the DEV Community article (which notes the same issue occurring with the Agent Toolkit skill) further supports the argument that this is a design-level issue rather than an isolated glitch.
  • As a temporary workaround, the most reliable approach is to use a standard OpenSearch Serverless collection (the previous generation) rather than a NextGen collection. While you will lose the cost-saving benefits of "scale-to-zero" capabilities, the traditional mechanism—which allows for explicitly specifying engine: faiss—is already proven to work with Bedrock integration.

answered 22 days ago

-2

Based on the information available about Amazon Bedrock knowledge bases with OpenSearch Serverless, there are specific requirements for the vector index configuration. When using OpenSearch Serverless as your vector store, the vector index must be configured with the faiss engine for compatibility with Amazon Bedrock knowledge bases.

However, it's important to note that the behavior you're experiencing with NextGen collections appears to differ from standard OpenSearch Serverless collections. The documentation indicates that if your vector index is configured with an incompatible engine, you would need to either:

  1. Create a new knowledge base in the console and let Amazon Bedrock automatically create a vector index in Amazon OpenSearch Serverless for you, or
  2. Create another vector index in the vector store with faiss as the engine, then create a new knowledge base specifying that new vector index

For NextGen collections specifically, if the collection is rejecting the explicit "engine" field entirely with a 400 error stating that the field parameter is not supported, this suggests there may be a difference in how NextGen collections handle engine configuration compared to standard OpenSearch Serverless collections.

I recommend reaching out through the AWS Support Center (https://console.aws.amazon.com/support/) to add correspondence about this specific behavior with NextGen collections, as this appears to be a configuration incompatibility that may require clarification on the proper index creation syntax for NextGen collections when used with Bedrock knowledge bases.
Sources
Sync your data with your Amazon Bedrock knowledge base - Amazon Bedrock
Modify a data source for your Amazon Bedrock knowledge base - Amazon Bedrock

answered 22 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.