- Newest
- Most votes
- Most comments
GPU support for Fargate/serverless compute, is not yet available, however it's in roadmap - refer here
As you have rightly mentioned, ECS and AWS Batch has good GPU support with VPC. Please refer below blogs/docs, in case that help in the challenges you are facing.
- Scheduling GPUs for deep learning tasks on Amazon ECS
- GPU workloads on AWS Batch
- Working with GPUs on Amazon ECS
- In case you are processing for Video Transcoding - Using video transcoding on Amazon ECS
An option might be to have a Lambda function that runs every minute. It checks the number of visible messages in the queue. If it increases, it start new ECS tasks, with GPU. The tasks themselves, run a loop that reads messages from the queue. If there are messages, it process them. If there are no more messages, the task exits.
You will need to do some tweaking as to when exactly to launch new tasks, i.e., how many messages should there be in the queue to increase the number of tasks.
Hi Uri, thanks for you answer.
Is the difference between AWS Batch and ECS only the fact that Batch also manage priority queues? I'm not sure I'm getting the differences between the services and which one match my needs...
As far as I understand, the solution you suggested me seems to be the best. Just to be sure, to implement this solution, I need to:
- Create an ECS cluster (with a minimum capacity of 0?)
- Create a task definition for my job requiring GPU
- Implement the lambda that will check the SQS queue and created the tasks depending on the number of messages in the queue
How is the scaling of the cluster working? Since GPU resources are pretty expensive, I want to be sure to exactly fit my needs. Does ECS knows how to scale down once the tasks are done?
Thanks!
Relevant content
- asked 4 months ago
- asked a year ago
- AWS OFFICIALUpdated 5 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 8 months ago
- AWS OFFICIALUpdated a year ago
Thanks! I will explore this links.
Any ETA regarding GPU support on Fargate? Any chances we'll get Lambda with GPU anytime soon?
For Fargate you can follow above listed github roadmap item, it's under WIP - refer at https://github.com/aws/containers-roadmap/projects/1