Skip to content

Opensearch Serverless Search OCUs are constantly hitting the maximum (32 OCUs)

0

Version: Amazon Opensearch Serverless

Issue:

I’m running into an issue with Amazon OpenSearch Serverless where my Search OCUs are constantly hitting the maximum (32 OCUs) which is 192 GB of RAM, even though I am not actively running any search queries and we do not have any users yet.

Configuration:

Collection type: Vector database Data size: ~115 GB (vectorized data + metadata) Number of shards: 2 Data was fully indexed around 2 weeks ago Indexing OCUs are stable and not an issue Only the Search OCUs spike continuously, leading to very high costs

Relevant Screenshots:

search OCUs

1 Answer
0

Based on your description, there are several factors that could be causing your Search OCUs to remain at maximum capacity despite minimal query activity.

First, it's important to understand that OpenSearch Serverless maintains a minimum baseline of OCUs even without active queries. When you create your first collection, OpenSearch Serverless instantiates a total of four OCUs (two for indexing and two for search). These OCUs exist even without indexing or search activity.

For vector search collections specifically, there are some unique considerations. Vector search collections keep index data in OCU local storage and are constrained by RAM space. They also cannot share OCUs with search or time series collections, even if they use the same KMS key. When you create your first vector collection, a new set of OCUs is created specifically for it.

The high OCU usage despite minimal query activity could be related to:

  1. Data size and complexity: Your 115 GB of vectorized data may require significant memory resources to maintain in a search-ready state, especially for vector search which is memory-intensive.

  2. Collection configuration: The number of collections you have can impact OCU usage. Some users have reported that reducing the number of collections (keeping only critical ones) helped decrease their SearchOCU count.

  3. Vector search specifics: Vector search collections have different scaling characteristics than standard search collections due to how they store and process data.

To address this issue, you could:

  • Monitor CloudWatch metrics like SearchOCU and SearchRequestRate to better understand usage patterns
  • Review if your collection configuration is optimal
  • Consider if your data could be organized differently to reduce memory requirements
  • Adjust your maximum OCU settings based on actual usage patterns

If the issue persists without clear reason, it might be worth contacting AWS support for a more detailed analysis of your specific configuration.
Sources
Managing capacity limits for Amazon OpenSearch Serverless - Amazon OpenSearch Service
OpenSearch Serverless OCU usage too high | AWS re:Post
AWS OpenSearch SearchOCU keeps hitting the max limit | AWS re:Post
Opensearch Serverless scale related questions | AWS re:Post

answered 3 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.