Managed Apache Flink - KPUs too high and not related to CPU usage

0

According to AWS docs:

*You are charged an hourly rate based on the number of Kinesis Processing Units (KPUs) used to run your applications. A single KPU comprises 1 vCPU compute and 4 GB of memory. *

I have several Flink jobs running in Managed Apache Flink on AWS. They have auto scaling enabled. All jobs run the same code. The problem is there seems to be no correlation between the CPU usage and the KPUs that AWS decides to provide my jobs. According to aws docs: your application will scale up when containerCPUUtilization is larger than 75 percent or above for 15 minutes.

For one of the jobs the "container cpu utilization" recently dropped from around ~25% to ~15% when the "kpu" metric shot up from 27 to 51 units. How can I lower my KPUs for these jobs? I do not believe they really require this amount of processing power. Below is a photo showing two jobs that run the same code however have wildly different cpu usage, and the same kpus.

Enter image description here

Enter image description here

No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions