Drupal deployment on multiple EC2 instances

0

We have a bunch of project hosted on EC2 instances. During the deployment we have a bunch of scripts being run. One of them is running the database updates as long as the drush configuration import commands. Now: usually the scripts runs on the second instance where the new code is deployed, so all is good. Both instances are ready with the new module we're installing for example, so all goes fine. In some occasions though we have seen the script being run on the first instance where the new code is deployed. This means that the other instance can't find new classes, so throws errors.

Has anyone found a solution for this? the documentation talks about the leader_only parameter i could use on my .ebextensions commands, but doesn't mention is the leader instance the last one touched by the new code, or is there any logic to choosing the leader?

Another solution would be removing this automated script and running it manually i guess... but i'd prefer our developers not having to remember doing manual things after the deployment is done :)

fede
asked 7 months ago262 views
2 Answers
0

Hi,

I'm Zainub from AWS Premium Support and it will be my pleasure assisting you.

From the correspondence, I understand that new application revisions are deployed through Elastic Beanstalk (EB). As part of the deployment, the database is updated. I understand that the database needs to be updated in order for the instances to successfully complete their deployment. To do this, the leader_only flag is set. The database script is usually executed on a specific instance (the second instance), however, sometimes it is executed on the first instance which causes the deployments to fail. Kindly confirm if the summary and understanding of the issue is correct.

Setting the "leader_only" flag means that the command is executed on a single instance chosen by Elastic Beanstalk. Leader-only container commands are run before other container commands. Further, the instance with the "leader_only" flag changes with each deployment and it is not assigned to any specific instance. This is because the ASG is constantly managing (launching and terminating) instances for various reasons, therefore, it is impossible for EB to know which instance you need to be a leader.

It appears that the application has very specific needs during deployment. Therefore, a custom solution is required to perform database updates during a deployment. Kindly note that assisting with custom solutions falls beyond the level of support generally provided. That said, I am happy to assist on a best effort basis.

There isn't enough information to give a specific answer, example, can the database update script be executed outside of EB? If so, consider using a CloudWatch event rule to trigger a database update script via Lambda when a new deployment is triggered for a particular environment. If the script needs to be executed on an instance, can all instances be given access to the required information to do the update? This will allow the chosen leader to execute the database update script successfully. These are only some examples of solutions that the application development team on your end can look into. It will need to be further inspected, tested and fleshed out by the team.

To summarize, the functionality required (requiring a specific instance to constantly be leader) is not supported by EB and a custom solution will need to be implemented in order to support the application needs.

I hope you find the above informative and have a great day ahead.

AWS
SUPPORT ENGINEER
answered 7 months ago
0

Thank you for you reply! It surprises me this is seen as a very specific need, because when the new code hits an instance, how is the other instance "reacting" to it? I mean:

  • code gets to instance A. It is the leader, so a script executes and sets table_a to have a new field
  • instance B gets a request. It still doesn't have the code to handle the new field on table_a, so it fails

Instead what i'd like to do is:

  • code gets to instance A.
  • code gets to instance B.
  • the script is run and sets the new field on table_a

we're happy sailing. I like your idea of the lambda triggering the DB update, but i wonder how is it called?

Maybe i'm just missing something obvious: is there an instance update schema where the build steps are mentioned? maybe there is something that is always executed at the end of a deployment?

fede
answered 7 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions