When should I consider reducing JVM heap size on Amazon OpenSearch Service for better search performance?

Understanding the JVM Heap and Compressed Object Pointers

By default, Amazon OpenSearch Service uses 50% of an instance's RAM for the JVM heap, up to a heap size of 32 GiB. This means that even on larger instance types with substantial RAM — such as r7g.4xlarge (128 GiB RAM) or r7g.12xlarge (384 GiB RAM) — the default heap allocation remains at 32 GiB. The remaining memory is left available for the operating system page cache, which OpenSearch relies on for fast reads of Lucene index segments.

However, in some cases the heap may be configured above this 32 GiB threshold — for example, through Auto-Tune recommendations or service-side optimizations for specific workload patterns. When this happens, it's important to understand the trade-offs.

The JVM uses an optimization called compressed ordinary object pointers (compressed oops) when the heap is at or below approximately 32 GiB. This optimization reduces the memory footprint of object references, allowing more objects to fit in the same amount of heap space and reducing garbage collection overhead.

When heap exceeds this threshold:

• Object pointers require more memory per reference
• The JVM needs to scan a larger heap during garbage collection, which can increase GC pause duration
• Less system memory remains available for the operating system page cache, which OpenSearch relies on for fast reads of Lucene index segments

This is a well-documented JVM behavior described in the OpenSearch documentation and Elasticsearch community guidance.

Workloads That May Benefit from a Smaller Heap

Not all workloads benefit from reducing heap. Consider your primary workload pattern:

Smaller heap (≤ 32 GiB) tends to benefit:

• Search-heavy workloads with frequent query execution
• Vector search / k-NN workloads that require off-heap memory for vector indices
• Mixed workloads where search latency is the primary concern
• Domains where garbage collection pauses correlate with latency spikes

Larger heap may be more appropriate for:

• Log analytics and write-heavy workloads with high bulk ingestion rates
• Domains with a very large number of shards per node
• Workloads with complex aggregations on high-cardinality fields
• Scenarios where you frequently encounter circuit breaker exceptions

CloudWatch Metrics to Evaluate

Before making any changes, review the following Amazon CloudWatch metrics for your domain over a representative time period (at least several days covering peak traffic):

JVMMemoryPressure — If this metric stays consistently low, your heap may be over-provisioned relative to your workload's needs.

JVMGCOldCollectionCount and JVMGCOldCollectionTime — Frequent or long-duration major GC events may indicate the JVM is spending significant time managing a large heap. If these events correlate with search latency increases, reducing heap size could help.

SearchLatency and SearchRate — If you observe high search latency despite low CPU utilization, the root cause may be GC pauses or insufficient OS page cache rather than compute capacity.

IndexingRate — If your domain is primarily write-heavy with a high indexing rate, a larger heap is more likely appropriate.

SysMemoryUtilization — High system memory usage combined with a large JVM heap may indicate the OS page cache is starved, which impacts Lucene's ability to serve search requests from cached segments.

General Decision Guidance

Ask yourself these questions:

1. What is my primary workload? If search or vector search is your main use case, a smaller heap generally allows more memory for OS page cache, improving read performance.

2. Is my JVM memory pressure consistently low? If your heap is rarely under pressure, you may be allocating more heap than your workload requires.

3. Do I see GC pauses correlating with latency spikes? This is a strong signal that GC overhead from a large heap is impacting your search performance.

4. Do I rely on large shard counts or heavy bulk indexing? If yes, be cautious — reducing heap without evaluation could lead to circuit breaker exceptions or out-of-memory scenarios.

How to Check Your Current Heap Configuration

You can check the current heap allocation on your nodes using the OpenSearch _cat/nodes API:

GET _cat/nodes?v&h=name,heap.max,heap.percent

This shows the maximum heap allocated to each node and current utilization.

How to Adjust JVM Heap Size

Amazon OpenSearch Service provides Auto-Tune to adjust JVM settings based on your cluster's performance metrics. Auto-Tune can recommend and apply heap size changes automatically.

If Auto-Tune does not address your specific needs, or if you require a custom heap configuration that is not available through the console, contact AWS Support to work with the OpenSearch Service team. They can evaluate your workload and assist with applying the appropriate JVM heap configuration for your domain.

Important:

• JVM heap size changes require a blue/green deployment. Plan accordingly and schedule during low-traffic periods.
• Test changes in a non-production environment first when possible.
• Monitor your domain closely after any change, watching for increases in JVM memory pressure, GC frequency, or thread pool rejections.

Key Takeaway

Bigger instances do not always mean you need more JVM heap. The optimal heap size depends on your workload type, not just your instance size. For search-intensive workloads, keeping heap at or below the compressed oops threshold allows the JVM to operate more efficiently and frees system memory for Lucene's file system cache — often resulting in better query performance than a larger heap would provide.

Related Information

• Auto-Tune for Amazon OpenSearch Service
• Troubleshoot high JVM memory pressure in OpenSearch Service
• Choosing instance types and testing
• OpenSearch performance tuning