MySQL RDS Replica Promotion Stuck in "Backing-Up" State

Question

Good evening,

We are in the middle of upgrading one of our RDS instances from MySQL 5.6 to MySQL 5.7. To minimize downtime, we took the path of creating a replica, upgrading that replica to 5.7, and then promoting it to the primary. The issue is after we selected to promote the database, its stuck in "Backing Up" state and has been for the past hour.  Before the promotion, we placed the primary instance into "Read Only" mode via a parameter group setting.

There is no indication that anything is happening, and the log files aren't updating either. The DB instance itself is still responsive to traffic.

Is this normal? AWS documentation said the promotion process would take "a few minutes". Snapshots/backups on our current production database only take about 2 minutes to complete...

theist · Accepted Answer

`backing-up` state means the database is available but it is taking a snapshot. So maybe the promotion is already done and the database is available for use. The replicas normally do not have backups enabled and it is mandatory for primaries to have them, so once a replica gets promoted it starts taking a backup, and this operation can take time as the first backup needs to copy the entire disk snapshot. `backing-up` instances are usable but not modifiable in AWS console.

After the first snapshot, the rest of them are faster, like in your production database, because consecutive snapshots only copy the disk differences between snapshots.

I've found that sometimes enabling backups and taking a manual snapshot of the replica meant to be promoted before the promotion speeds up any automated backup created later, but YMMV.

MySQL RDS Replica Promotion Stuck in "Backing-Up" State

Relevant content