How do I automatically evaluate and remediate the increasing volume on an Amazon EC2 instance when free disk space is low?

Short description

To evaluate and extend Amazon Elastic Block Store (Amazon EBS) volumes, use a set of AWS Systems Manager Automation documents. The Automation documents allow you to investigate and optionally remediate low disk usage on an Amazon EC2 instance.

Based on the OS type, the AWSPremiumSupport-TroubleshootEC2DiskUsage Automation document directs the run of the other Systems Manager documents.

The following documents perform basic diagnostics and evaluate whether you can expand your volume size to migrate the volumes:

• AWSPremiumSupport-DiagnoseDiskUsageOnWindows
• AWSPremiumSupport-DiagnoseDiskUsageOnLinux

The following documents take the output of the preceding documents and run Python code to perform the volume modification:

• AWSPremiumSupport-ExtendVolumesOnWindows
• AWSPremiumSupport-ExtendVolumesOnLinux

The automation then accesses the instance and extends the partition and file system of the volumes.

Resolution

Grant permissions

You must grant the following permissions to use the Automation documents. First, make sure that you have an AWS Identity and Access Management (IAM) instance profile for Systems Manager that's attached to the target instance.

To grant permissions, complete the following steps to set up AssumeRole to specify the AutomationAssumeRole parameter for the Automation document configuration process:

1. Use the JSON option to create the following policy:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Action": [
           "ec2:DescribeVolumes",
           "ec2:DescribeVolumesModifications",
           "ec2:ModifyVolume",
           "ec2:DescribeInstances",
           "ec2:CreateImage",
           "ec2:DescribeImages",
           "ec2:DescribeTags",
           "ec2:CreateTags",
           "ec2:DeleteTags"
         ],
         "Resource": "*",
         "Effect": "Allow"
       },
       {
         "Action": [
           "iam:PassRole"
         ],
         "Resource": "*",
         "Effect": "Allow"
       },
       {
         "Action": [
           "ssm:StartAutomationExecution",
           "ssm:GetAutomationExecution",
           "ssm:DescribeAutomationStepExecutions",
           "ssm:DescribeAutomationExecutions"
         ],
         "Resource": "*",
         "Effect": "Allow"
       },
       {
         "Action": [
           "ssm:SendCommand",
           "ssm:DescribeInstanceInformation",
           "ssm:ListCommands",
           "ssm:ListCommandInvocations"
         ],
         "Resource": "*",
         "Effect": "Allow"
       }
     ]
   }

2. Create the assume role, and then attach the policy.

3. Modify the following statement. Replace "Resource": "*" with your ARN for the assume role:

   {
     "Action": [
         "iam:PassRole"
       ],
       "Resource": "*",
       "Effect": "Allow"
     },

Run the Automation document

To use the set of Systems Manager Automation documents, run only the AWSPremiumSupport-TroubleshootEC2DiskUsage document.

To run the AWSPremiumSupport-TroubleshootEC2DiskUsage document, complete the following steps:

1. Open the Systems Manager console, and then choose Automation in the navigation pane.
2. Choose Execute automation.
3. Select AWSPremiumSupport-TroubleshootEC2DiskUsage, and then choose Next.
4. For Execute automation document, choose Simple execution.
5. Under Input parameters, enter the following information:
   For InstanceId, enter your Amazon EC2 instance ID.
   For AutomationAssumeRole, enter the ARN of the assume role to allow the Automation to perform the actions on your behalf.
6. Under Input parameters, specify the following inputs if your requirements differ from the default values:
   VolumeExpansionEnabled: Controls whether the document extends the affected volumes and partitions (default: True)
   VolumeExpansionUsageTrigger: Minimum percentage of used partition space that's required to trigger expansion (default: 85)
   VolumeExpansionCapSize: Maximum size in GiB that the EBS volume increases to (default: 2048)
   VolumeExpansionGibIncrease: Volume increase in GiB (default: 20)
   VolumeExpansionPercentageIncrease: Volume increase in percentage (default: 20)
7. Choose Execute.

Example scenario

Your current volume is 30 GB and has 4 GB free. This means that you have 26 GB of used space. Specify the following input parameters:

• VolumeExpansionUsageTrigger: 85
• VolumeExpansionGibIncrease: 10
• VolumeExpansionPercentageIncrease: 15
• VolumeExpansionCapSize: 2048

Because 26 GB of used space is above the 85% threshold for VolumeExpansionUsageTrigger, a 10 GB volume increase is initiated. This increase occurs because you specified that the volume increases by either 10 GB or by 15% of the current volume size of 4.5 GB. The Automation document uses the biggest net increase between VolumeExpansionGibIncrease and VolumeExpansionPercentageIncrease. The new volume size is 40 GB. 40 GB is within the specified 2048 VolumeExpansionCapSize.

Related information

Extend a Linux file system after resizing a volume

Extend a Windows file system after resizing a volume

Use IAM to configure roles for Automation