How can I send memory metrics from a fleet of EC2 instances to CloudWatch?

Amazon EC2 emits several metrics for your EC2 instance to Amazon CloudWatch. However, memory metrics is not one of the default metrics provided by Amazon EC2. Using CloudWatch Agent, we can capture memory metrics. With memory metrics, we can –

• improve right sizing recommendation of EC2 Instances
• scale memory heavy applications like big data analytics, in-memory databases, real-time streaming with memory utilization

Memory metrics include Active Memory, Available Memory, Buffer Memory, Cached Memory, Free Memory, Inactive Memory, Total Memory, Used Memory.

To capture memory metrics from a fleet of EC2 instances, we need to -

1. setup (install, configure, start) CloudWatch Agent on one of the instances (either using CLI or AWS Systems Manager)
2. setup on other instances of the fleet (using AWS Systems Manager)

Setup CloudWatch Agent using CLI on first instance

1. Install CloudWatch Agent
2. Configure CloudWatch Agent using the Configuration Wizard, and store config.json in AWS Systems Manager ParameterStore (or, manually add JSON in ParameterStore). If Configuration Wizard is used CloudWatchAgentAdminPolicy permission policy needs to be added to EC2 Instance Role. By default, only 'mem_used_percent' metric is included in the configuration file. We need to manually edit the JSON file to include other metrics (mem_available_percent, mem_available, mem_total, mem_cached, mem_active, mem_inactive, mem_free, mem_used, mem_buffered)
3. Add CloudWatchAgentServerPolicy permission policy on the EC2 Instance Role, to allow CloudWatch Agent to read from EC2 instance & write metrics to CloudWatch.
4. Start CloudWatch Agent with the configuration file

Setup CloudWatch Agent on other instances of the fleet using AWS Systems Manager

1. Install CloudWatch Agent on all needed EC2 instances using AWS Systems Manager - Quick Setup - Host Management
2. Add CloudWatchAgentServerPolicy permission policy on all the EC2 Instances' Roles (using a custom script)
3. Start CloudWatch Agent with the configuration file stored earlier in AWS Systems Manager Parameter Store, using AWS Systems Manager - Run command (use 'AmazonCloudWatch-ManageAgent' command). Note that Parameter Store is a regional service, so if we have EC2 instances in more than a region, we need to setup a parameter in those region(s).

Please refer to https://repost.aws/knowledge-center/cloudwatch-memory-metrics-ec2 for more details on how to capture memory metrics from a EC2 instance, using CLI.