Can't update MSK config due to number of partitions

Hi!

As you are way below the recommended partition level I would recommend to open a support ticket to see whether they help.

Bests!

Please make sure you are counting all partitions (with their replicas), not only partition leaders. It is very possible that the numbers you are presenting are leaders only per broker, and a total number would be about as 3 times per broker, which does exceed the limit. If that is not the case, go ahead and open AWS Support ticket.

Since you are using kafka.m5.large instance type for your brokers, the maximum recommended number of partitions per broker is 1000. Thus, I would request you to check the partition count at broker level. You can check it in the Kafka CloudWatch metrics (ClusterName, brokerId)with metric PartitionCount. This will show you the exact number of partitions on each of your broker including the replica and leader brokers. If you are reaching the maximum limit of 1000 for any of your broker, the exception that you are getting is expected.

So, if the partition count is exceeding, you would need to take any of the below actions to reduce the partition count per broker.

1. Increase the number of brokers within the cluster and then reassign partitions to reduce the number of partitions per broker

2. Decrease the number of partitions by deleting unused topics

3. Modify instance type to higher instance type. For example kafka.m5.2xlarge(if possible) since it has a recommended upper limit of 2000 partitions per broker.

With that being said, I would request you to follow any of the above steps to make sure the partition count is within the limit[1] according to your broker type. After doing that, try to change the configuration you required.

However, if you are not exceeding the partition count 1000 on all your brokers and still you are not able to update the Configuration, I would request you to reach out to us over a Support Case.