- Newest
- Most votes
- Most comments
Update on the answer, the "defaults" settings are not able to be changed but the "persistent" changes are able to be changed, the correct thing to do to evict and reset the KNN cache is to perform the following request:
HTTP PUT /_cluster/settings {"persistent":{"knn.memory.circuit_breaker.limit": null } }
Hi, As you mentioned yo need to reset the k-NN circuit breaker when it is triggered in Amazon OpenSearch Service, and the k-NN circuit breaker is controlled by the knn.memory.circuit_breaker.limit cluster setting, with this setting you can determine the maximum amount of memory that can be used by the k-NN feature.
A "denied access" error when trying to update this setting, could mean that you do not have the necessary permissions to modify the cluster settings. You will need to request the appropriate IAM permissions from your AWS administrator. The required permissions are: "opensearch:UpdateDomainConfig" "opensearch:DescribeDomain" "opensearch:DescribeDomainConfig"
After you update the setting, the k-NN circuit breaker should be reset, and your queries should be able to execute without triggering the circuit breaker.
Relevant content
- Accepted Answerasked 9 months ago
- Accepted Answerasked a year ago
- AWS OFFICIALUpdated a month ago
- AWS OFFICIALUpdated 5 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 6 months ago
Thank you for your response, however I don't believe that to be the issue after trying it and confirming all opensearch permissions are set on the cluster which has IP Address level security configuration.
The IAM permissions are added and the response from the cluster is the same response. According to this: https://forum.opensearch.org/t/how-to-deal-with-knn-circuit-breaker-triggered-stays-set-nodes-at-max-cache-capacity/4733
This request should return the value to its default and trigger a "cache rebuild where all entries are evicted".
Request Body: HTTP PUT /_cluster/settings {"default":{"knn.memory.circuit_breaker.limit": null } }
Response Body {"Message": "Your request: '/_cluster/settings' payload is not allowed."}
If this cannot be set by some feature of OpenSearch then is there another way to cause a "cache rebuild where all entries are evicted" safely?