Looking for an ecs/eks fargate + cloudwatch + sqs example

0

Hello all,

I'm looking for a complete working example for ecs/eks fargate + cloudwatch + sqs architecture (see Fig 5) that is described in this article - https://aws.amazon.com/blogs/architecture/rate-limiting-strategies-for-serverless-applications/

Any pointers is very much appreciated.

2 Answers
1

Hi,

The AWS Cloud Development Kit (CDK) provides various reusable Constructs as basic "building blocks". The CDK does provide a QueueProcessingFargateService construct? This construct helps configure the SQS Queue to poll and process, the Autoscaling capacity of the ECS Fargate service, and the integration.

Other CDK Patterns are available at:

Hope it helps point you in the right direction! Thanks

AWS
answered 2 years ago
1

Hello. For a more generic answer and not a CDK one: this is a very common pattern that I have used many times, to the point that we have an off-the-shelf solution for it. Alike CDK, it uses CFN too and docker-compose, via using ECS Compose-X to configure queue, its DLQ, and auto-scale services in ECS (compute platform in Compose-X defaults to Fargate, but you can choose whatever suits you, I use it at home too with ECS Anywhere).

There is an example repository with the compose files here which is that off the shelves solution I mentioned. This is a very simple workflow: S3 -> new file / file update -> SQS -> CW Alarm -> Autoscaling of an ECS Service -> process files -> done.

What I like about this approach vs S3 -> Lambda directly is that using SQS "as the carrier" any service, anywhere, on any compute, can process the messages in the same way (Lambda, ECS, EC2, on-prem, so long as it has access to the messages). In some cases we even implemented that based on filesize, using the same code, some files were processed in Lambda, some with ECS.

Anyways, regardless, the idea is that you can work on your service using a combination of docker-compose, possibly add ECS Local Endpoints too for IAM high fidelity, define your services, define your resources, link them logically in a "no-code" fashion and simply deploy.

Hope this helps too,

profile picture
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions