Radial search not working for Opensearch Serverless index, how to make it work?

Question

**max_distance** or **min_score** arguments are not working in opensearch serverless index. It's raising "failed to create query: [knn] requires exactly one of k, distance or score to be set" exception while tried using both Opensearch Dev Tool and 'opensearch-py' Python library. Here's the detail of the index and api actions:

```
PUT knn-index-test
{
  "settings": {
    "index.knn": true
  },
  "mappings": {
    "properties": {
      "my_vector": {
        "type": "knn_vector",
        "dimension": 2,
        "method": {
            "name": "hnsw",
            "space_type": "l2",
            "engine": "faiss",
            "parameters": {
              "ef_construction": 100,
              "m": 16,
              "ef_search": 100
            }
          }
      }
    }
  }
}

POST _bulk
{"index": {"_index": "knn-index-test"}}
{"my_vector": [7.0, 8.2], "price": 4.4}
{"index": {"_index": "knn-index-test"}}
{"my_vector": [7.1, 7.4], "price": 14.2}
{"index": {"_index": "knn-index-test"}}
{"my_vector": [7.3, 8.3], "price": 19.1}
{"index": {"_index": "knn-index-test"}}
{"my_vector": [6.5, 8.8], "price": 1.2}
{"index": {"_index": "knn-index-test"}}
{"my_vector": [5.7, 7.9], "price": 16.5}

GET knn-index-test/_search
{
  "query": {
    "knn": {
      "my_vector": {
        "vector": [7.1, 8.3],
        "max_distance": 2,
        "filter": {
          "range": {
            "price": {
              "gte": 1,
              "lte": 5
            }
          }
        }
      }
    }
  }
}
```

And, here's the error:

> {
  "error": {
    "root_cause": [
      {
        "type": "query_shard_exception",
        "reason": "failed to create query: [knn] requires exactly one of k, distance or score to be set",
        "index": "123456789::mvrwb2glh8irm8uws43l::VECTORSEARCH::knn-index-test:0",
        "index_uuid": "R2X6_JMBRx_DgNPiAH85:0"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": []
  },
  "status": 400
}
>

Documentation followed: 
[https://opensearch.org/docs/latest/search-plugins/knn/radial-search-knn/]()

re:Post Agent · Answer

The issue you're experiencing with radial search not working in OpenSearch Serverless is likely due to the fact that OpenSearch Serverless currently does not support the radial search feature. Radial search, which allows you to search points in a vector space within a specified maximum distance or minimum score threshold, is a feature that was introduced in OpenSearch version 2.15 for provisioned domains.

However, OpenSearch Serverless may not have this feature implemented yet. Serverless offerings often lag behind in terms of feature parity with their provisioned counterparts.

To make your query work in OpenSearch Serverless, you'll need to modify it to use the standard k-nearest neighbor (k-NN) search instead of radial search. Here's how you can adjust your query:

1. Replace the `max_distance` parameter with `k` (number of nearest neighbors to return).
2. Remove the `filter` section from within the `knn` query (as it's not supported in this context).

Your modified query should look like this:

```
GET knn-index-test/_search
{
  "query": {
    "knn": {
      "my_vector": {
        "vector": [7.1, 8.3],
        "k": 5
      }
    }
  }
}
```

This query will return the 5 nearest neighbors to the given vector [7.1, 8.3].

If you need to filter results based on the `price` field, you can use a `bool` query to combine the k-NN search with a range filter:

```
GET knn-index-test/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "knn": {
            "my_vector": {
              "vector": [7.1, 8.3],
              "k": 5
            }
          }
        }
      ],
      "filter": [
        {
          "range": {
            "price": {
              "gte": 1,
              "lte": 5
            }
          }
        }
      ]
    }
  }
}
```

This query will perform a k-NN search and then filter the results based on the price range.

Keep in mind that while this approach will work, it doesn't provide the exact functionality of radial search. You may need to retrieve more results (higher k value) and then post-process them in your application to achieve similar results to what radial search would provide.

For the most up-to-date information on feature availability in OpenSearch Serverless, it's recommended to check the official AWS documentation or contact AWS support.  
**Sources**  
[Amazon OpenSearch Service now supports OpenSearch version 2.15 - AWS](https://aws.amazon.com/about-aws/whats-new/2024/09/amazon-opensearch-service-version-2-15/)  
[Amazon OpenSearch Serverless - Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless.html)

Radial search not working for Opensearch Serverless index, how to make it work?

Add your answer

Relevant content