AWS Opensearch Serverless supports for refresh=wait_for and refresh=true

Question

I'm trying to set up a Opensearch Serverless Collection for our application. Our use case require us to wait for updating/indexing document to complete in Opensearch. Currently I'm using a Opensearch cluster and I am using refresh=wait_for and it' is working fine.

Upon trying with the Opensearch Serverless Collection we tried the same action but I got this error:

```
{
  "error": {
    "root_cause": [
      {
        "type": "status_exception",
        "reason": "wait_for refresh policy is not supported."
      }
    ],
    "type": "status_exception",
    "reason": "wait_for refresh policy is not supported."
  },
  "status": 400
}
```

Here is an example request:

```
POST test_index/_doc?refresh=wait_for
{
  "field1" : "value1",
  "field2" : "value2",
  "field3" : "value3"
}
```

Is this a limitation to Opensearch Serverless as a whole or is there some other workaround that I can use?

Answer

The error message you're seeing suggests that the **wait_for** refresh policy is not supported in your current setup, likely due to the serverless nature of the AWS OpenSearch Service.

Serverless options like AWS OpenSearch Service offer many benefits, but they also come with some limitations due to the abstraction and automatic scaling. Specifically, with serverless options, you do not manage the underlying infrastructure, and therefore certain operations that require explicit knowledge of that infrastructure might not be supported.

In the case of the r**efresh=wait_for** policy, this policy tells OpenSearch to wait for a refresh to make the changes searchable. This requires the system to have control over when a refresh happens, which may not be available in a serverless environment due to the automatic scaling and management.

Unfortunately, if this feature is not supported in the serverless version of OpenSearch, there might not be a direct workaround. However, you could attempt to manually enforce consistency in your application logic. For example, you could introduce a delay after making changes to an index before you attempt to read the data. This is not ideal because the delay would need to be long enough to account for potential variance in refresh times.

Ultimately, the decision between a traditional managed OpenSearch cluster and a serverless OpenSearch collection will depend on the specific needs of your application. If you need a feature like the **refresh=wait_for** policy, you might need to use a traditional managed OpenSearch cluster.

AWS Opensearch Serverless supports for refresh=wait_for and refresh=true

Relevant content