Opensearch query to return number of events over rate threshold

Question

For a given time window I would like to return the total number of events that occurred over a certain rate threshold in any given month, for example I have a load balancer receiving requests each request generates an event, if the threshold is 1 event per second how many events were there in any given calendar month where the rate of requests was higher than 1 per second over a 1 minute window in any minute of that month

I understand that the best way to solve this would be to create a rollup index that calculates this and provides the value every minute however this isn't helpful when trying to look retrospectively at data

Answer

Hi, Looks like you were trying to find out a way to do some 'alerting' system by using this query, if I understand correctly.

In Opensearch, there are several aggregation functions [1]but what you were looking for is a steady function that keeps reviewing the incoming event and pop up a message when the threshold is met.

To be more specifically, seems like you are looking for an analytics application that can query the streaming data, which is what Kinesis analytics application[2] does.

By the way, maybe you can try to use the Opensearch Dashboard[3]. Inside Dashboard there are some analytics metrics what can be used to do similar analytics.

I hope you find above information useful!

[1]https://opensearch.org/docs/latest/opensearch/aggregations/
[2]https://aws.amazon.com/kinesis/data-analytics/
[3]https://docs.aws.amazon.com/opensearch-service/latest/developerguide/dashboards.html

Opensearch query to return number of events over rate threshold

Relevant content