Automatically reboot EC-2 linux servers of a target-group if OS update requires a reboot


We're having some Ubuntu instances that are registered targets of Target Groups behind an Elastic Load Balancer. Also, those servers make use of the "unattended-upgrades" package to install security relevant packages. Some of those newly installed packages require the server to be rebooted. Therefore it sends an email to our System Engineers to let them know.

So now, in order to reboot those instances they need to be deregistered from their Target Group, rebooted, and registered again with the Target Group. Those Target Groups have redundant targets - so the missing one target is okay for the time it takes to make it become functional again.

So now my actual question. Can this easily be automated or is there some light-weighted solution available? If possible I would like to avoid a "full-blown" fleet management software. However, I can see how it can get complicated fast but still thought to ask.

My first thought was some sort of AWS-CLI scripting that unregisters the instance from the target-group and registers itself again after the reboot succeeded. If there are enough other targets available to cover for a few minutes. Or maybe have the instance shut down and let an auto-scaling group boot up a new instance. However, that new instance would need to be updated from the base-image first as well then.

Any idea where or what to look for?

Thanks, M

asked 10 months ago136 views
1 Answer

This could be accomplished with a few AWS services and some scripting. If your unattended upgrades sends an email to SysAdmins, you could leverage Amazon SES to accept the email and then trigger a Lambda function to deregister the target and reboot the server. Finally you could use AWS Step Function State Machines to monitor the status of the reboot and then register the target when the server is back up.

profile pictureAWS
answered 10 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions