How to schedule MSK security patches to run outside of business hours?

Question

Hello,

We recently experienced service instability due to automatic security patches being applied to our Amazon MSK cluster during peak business hours (approx. 10:00 AM local time). These updates triggered broker reboots, causing connection issues and lag for our producers and consumers.

While we understand the importance of security patches, running them during high-traffic periods is risky for our production environment.

My question is: Is there a way to strictly enforce a specific time window (e.g., early morning) for all MSK patching activities, including critical security updates? We want to ensure these operations only occur during our defined maintenance window to avoid business impact.

Any guidance on how to configure this or request a schedule change for these specific patches would be appreciated.

Aditya · Answer

Just wanted to know what was your Replication factor and Minimum In-Sync Replicas for your MSK cluster?

raresonal · Answer

You need to reach out to AWS Support team to request the reschedule of your maintenance window to non-production hours. Also, you can ask them to reschedule it to particular time and day of every month.

Ex: Every 1st Sunday at 02:00AM UTC

April_D · Answer

**Yes**, you can schedule MSK maintenance windows to run outside of business hours for Standard Brokers by contacting AWS Support. AWS Support can reschedule your cluster's maintenance window to align with your preferred time.

To request a maintenance window change, open a support case with AWS Support and provide the following information:

**New Maintenance Window Timeframe**:
- Week of the month (1st, 2nd, 3rd, or 4th)
- Day of the week (Mon, Tue, Wed, Thu, Fri, Sat, Sun)
- Time of the day in UTC

**Important notes about maintenance scheduling**:

The weeks correspond to specific date ranges: Week 1 (days 1-7), Week 2 (days 8-14), Week 3 (days 15-21), and Week 4 (days 22-28). Maintenance can only be scheduled between dates 1-28; the 5th week or dates after the 28th are not available options.

Once your new maintenance window is set, it will follow the same pattern for upcoming months until you submit another request to change it. For example, if you schedule maintenance for the 1st Sunday at 08:00 AM UTC, all future maintenance will occur on the 1st Sunday of each month at that time.

You cannot skip a maintenance window for a month—the maintenance must be scheduled within the current month.

**Note**: There are no maintenance windows for Express brokers: https://docs.aws.amazon.com/msk/latest/developerguide/patching-impact.html#patching-express-brokers

SHAJAM · Answer

I don't think there is a maintenance window for MSK. However, you can follow best practices to reduce downtime.
https://repost.aws/knowledge-center/msk-avoid-disruption-during-patching

Jeongho Kim · Answer

Regarding changing your maintenance window schedule, please note that security patching should have no impact on your applications' writes and reads if you follow best practices [1, 2].

On very rare occasions, a broker might become unhealthy, and Amazon MSK has workflows to replace the unhealthy broker. Focusing on fixing any issues noticed during security patching will also ensure that any unplanned single broker failure will have no impact on your applications' writes and reads. 
If you determine that changing your maintenance window schedule still remains necessary due to critical business events or operational requirements, please contact AWS Support.

[1] https://docs.aws.amazon.com/msk/latest/developerguide/patching-impact.html#patching-standard-brokers

[2] https://docs.aws.amazon.com/msk/latest/developerguide/bestpractices.html

How to schedule MSK security patches to run outside of business hours?

Add your answer

Relevant content