By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

How to configure Service Auto-scaling in AWS ECS (using step scaling)

0

We have a number of dotNet services running in ECS which consume data from RabbitMQ queues, I want to scale up the number of instances based on the number of messages in the queues.

Let's say I want to have one instance running if the queue is below 100 and multiple as the load increases. [The values here are for example purposes only]

I have created a CloudWatch Alarm which monitors the queue in question and goes into alarm when it hits 100.

I have enabled the service auto-scaling, set the min (1) and max (5) and created a scaling policy which monitors that alarm and sets the scaling actions as follows...

  • Add 1 task from 100 to +infinity

This appears to work and I see the instances gradually climb to five while the queue is 100+. What I cannot work out is how to scale down correctly. I have tried all manner of combinations to try to scale it down but just cannot get a satisfactory result.

Can someone please point me to a good example of how exactly to set this up. I've found a number of articles which skirt round the edges but can't find that one decent example of exactly how to configure this scenario.

Your help would be much appreciated.

  • Shahad_C
    6 months ago

    You mentioned Service AutoScaling, but also mentioned Instances a few times. Just confirming, is the issue here related to scaling the number of instances in an ECS cluster (For a non-Fargate based service), or the number of Tasks that Service AutoScaling (Application AutoScaling) is creating?

Topics
Management & GovernanceContainers
Tags
AWS Auto ScalingAmazon Elastic Container Service
Language
English
Mike
asked 6 months ago653 views
1 Answer
0
Accepted Answer

Good day, Mike. Great question and one that I don't think we focus enough on. I do have one question: Can you please let me know what your configuration looks like for the scaling in metric? You stated that once the RabbitMQ queue hits 100+, the scale out event is invoked and you start to see the number of instances increase. But once the RabbitMQ queue drops below 100, is there a desired state or rate of scale in based on the metric (something like 80 = 4 instances, 60 = 3 instances, etc.)? Based on what it looks like you're trying to accomplish, I'd recommend using something similar to what is described here with Monitoring MQ Brokers with CloudWatch and Monitoring RabbitMQ with CloudWatch. Then, based on those metrics (i.e. MessageReadyCount<=100) you can adjust your AutoScaling policy. Something else to consider would be Target Tracking Scaling Policies where you can select a metric and set a target value. Amazon ECS Service Auto Scaling creates and manages the CloudWatch alarms that control the scaling policy and calculates the scaling adjustment based on the metric and the target value.

Hopefully this points you in the right direction for being able to configure your scale in triggering event for ECS.

Thank you, Brian at AWS

profile pictureAWS
BrianDAnderson_AWS
answered 6 months ago
profile picture
EXPERT
Gary Mclean
reviewed a month ago
  • Shahad_C
    6 months ago

    A note for Target Tracking: the metric would need to be changed to be proportional to the number of tasks. Something like "messages per task", since target tracking requires metrics to change proportionally to the actual capacity being scaled. Here's a very similar article for doing this with SQS: https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-using-sqs-queue.html

  • Mike
    6 months ago

    Thanks Brian. As far as below 100 is concerned I'm happy for it to go back to 1 instance immediately as the message surge would have been catered for by then. I'll have a read of those today and see it I can create a solution. Cheers.

  • Shahad_C
    5 months ago

    Step scaling policies support a change type of "ExactCapacity" https://docs.aws.amazon.com/cli/latest/reference/application-autoscaling/put-scaling-policy.html - so you could have an alarm trigger a scale-in policy when task count is < 100 with a type of ExactCapacity and adjustment of 1 (maybe put a tiny bit of buffer to prevent oscillation when the queue is right around that 100 value, something like <90). On the console, do this by selecting "Set to" on the Action dropdown for the Scale-in policy

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content