Memory usage increase on ECS services after migrating to amazon linux 2023
Hello, We recently migrated our ECS clusters from Amazon Linux 2 to Amazon Linux 2023, and we encounter a massive increase in term of memory usage on all of our containers.
We use ECS on EC2 with ECS-optimized AMIs, no custom configuration, we changed nothing except the AMI.
As you can see here, memory usage on ecs services almost doubled right after the AMI change.
Which mechanically increases the memory usage on our EC2.
We can observe this behavior on all our services for all clusters (around 40 services, 12 clusters). Our applications still run correctly and we haven't observed any oom kills or any other issues for the moment, which leads me to believe this is more a matter of different memory management between the two OS rather than a real problem of overconsumption.
Is there anything that could explain this issue ?
- Newest
- Most votes
- Most comments
Hi,
Do you have old versions of some packages in the ECS container images ? Or are you running with up-to-date for each packages ?
I have seen similar issues due to the fact that old versions of some packages where not working well with a newer Linux kernel.
Best,
Didier
Relevant content
- Accepted Answerasked a year ago
- asked a year ago
AWS OFFICIALUpdated 4 months ago- AWS OFFICIALUpdated 6 months ago
- AWS OFFICIALUpdated 2 months ago
Thanks for your answer. Our base images are "amazoncorretto:21.0.3-al2023" or "amazoncorretto:21.0.2-al2023" for most of our containers, the rest is essentially (a pretty much up to date) node:alpine.
I'm (almost) certain we don't use packages older than a few months