DocumentDB ChangeStream, Performance Insights and high Database Load

0

Hi, I have an application that connects to docDB and start listening to the changestream, but Performance Insights is showing high Database Load the moment we start watching the changestream, like shown above: Enter image description here Enter image description here

I would like to know the best way to read this chart. Apparently the changestream utilizes all CPUs from the instance, even when the database is idle and not receiving any queries. Do I need to worry about that high Database Load? Will enabling and reading the changestream make my database slower in general? What would be the effects on performance? AWS docs seen to be lacking on this information.

asked a year ago536 views
1 Answer
0

Dear AWS User,

I understand that you are observing high CPU usage on your DocumentDB cluster the moment your Application begins to read from the ChangeStreams even when the cluster is relatively idle. From the graphs, I do see that we are maxing out in terms of vCPUs on the primary instance and hence we will need to identify the queries reported at Performance Insights (PI) to have more visibility on the issue.

While the usage of ChangeStreams does consume some resources it shouldn't necessarily lead to maxing out of CPUs and we will need to look over the associated queries.

When you attempt to get the full query from PI dashboard, there is a probability that it will get truncated. This is because by default, each row in the Top queries table shows 500 bytes of query text for each query statement. When a digest statement exceeds 500 bytes, you can view more text by opening the statement in the Performance Insights dashboard. In this case, the maximum length for the displayed query is 1 KB.

If you want to view a full query statement, you can also choose Download [ 1 ] [ 2 ].

Kindly note that performance insights does not an option to map a specific query to its correlating opid. In light of this, you can find it useful using the following step sequence in helping identify and mitigating queries that cause high CPU issues here [ 3 ].

These will help you to get more insights into what queries are leading to the excessive CPU usage. Then accordingly, I will advice you to followup with your DBAs/ AWS Support Team via a support ticket to troubleshoot more on the issue.

Have a great and wonderful day ahead!

Reference links:

[ 1 ]: Overview of the Top queries tab - Accessing statement query text - https://docs.aws.amazon.com/documentdb/latest/developerguide/performance-insights-top-queries.html#performance-insights-top-queries-accessing-text

[ 2 ]: Overview of the Top queries tab - Viewing and downloading statement query text - https://docs.aws.amazon.com/documentdb/latest/developerguide/performance-insights-top-queries.html#performance-insights-top-queries-viewing-downloading

[ 3 ]: Performance and Resource Utilization - How Do I Determine the Cause of High CPU Utilization on One or More Cluster Instances? - https://docs.aws.amazon.com/documentdb/latest/developerguide/user_diagnostics.html#user_diagnostics-cpu_utilization

AWS
SUPPORT ENGINEER
Mano_S
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions