Why can't my IAM user or role with full Amazon EC2 permissions start an EC2 instance?

Resolution

Determine the cause of your EC2 instance's Stopped state

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

If you attached an Amazon Elastic Block Store (Amazon EBS) volume to your EC2 instance, then check the volume encryption. If you encrypted the EBS volume with an AWS Key Management Service (AWS KMS) key, then there might be a permission issue. The IAM principal that calls the StartInstances API action must have kms:CreateGrant permissions to create a grant for Amazon EC2. The grant allows Amazon EC2 to decrypt the volume's data key with your AWS KMS key.

EBS volumes send a GenerateDataKeyWithoutPlaintext API call to AWS KMS that creates a new data key and encrypts it in the AWS KMS key. AWS KMS sends the encrypted data key to the EBS volume. Then, the volume attaches the data key to the instance. The data key is present in the same AWS account as the instance and AWS KMS key.

To determine why the instance is in a Stopped state, run the describe-instances AWS CLI command:

aws ec2 describe-instances --instance-id your-instance-id --query "Reservations[*].Instances[*].StateReason"

Note: Replace your-instance-id with your instance id.

Example output:

[  
  [
    {
      "Message": "Client.InternalError: Client error on launch",
      "Code": "Client.InternalError"
    }
  ]
]

The preceding error might mean that you encrypted the root volume or additional attached volumes. You don't have permission to access the AWS KMS key for decryption.

You can also filter AWS CloudTrail events for the event name CreateGrant.

Example output:

"errorMessage": "User: arn:aws:iam::123456789012:user/test is not authorized to perform: kms:CreateGrant on resource: arn:aws:kms:eu-west-1:123456789012:key/8e3426b8-87b4-434c-ae74-8e63dadf354a"

The preceding error means that the IAM principal doesn't have the CreateGrant permission for Amazon EC2 to decrypt the data key. So, the instance can't start.

Note: If you use an AWS managed key, then this error only occurs if there's an explicit deny in the principal's IAM policies or service control policy (SCP). AWS managed keys don't require additional IAM permissions for use.

Find the AWS KMS key type

Complete the following steps:

1. Open the Amazon EC2 console.
2. In the navigation pane, choose Instances.
3. Select the instance ID, and then choose Storage.
4. For Volume ID, select the volume ID for the encrypted volume.
5. For KMS key ID, copy the AWS KMS key ID.
6. Open the AWS KMS console in the same AWS Region.
7. In the navigation pane, choose Customer managed keys. Then, search for the AWS KMS key ID from step 4.
8. Select the AWS KMS key ID.
9. In General configuration under Description, note the AWS KMS key type.

Attach an IAM policy

Attach an IAM policy to the IAM principal similar to the following example:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "kms:CreateGrant"
      ],
      "Resource": [
        "arn:aws:kms:region:123456789012:key/ExampleKey"
      ],
      "Condition": {
        "Bool": {
          "kms:GrantIsForAWSResource": true
        }
      }
    }
  ]
}

The kms:GrantIsForAWSResource condition key assures that the IAM principal only creates grants for the AWS KMS key with AWS resources. This policy doesn't allow the IAM principal to create grants for another IAM principal. To restrict access to Amazon EC2, use the kms:ViaService condition key.

You must activate IAM permissions that you add to the IAM principal through the key policy for your account, as shown in the following example:

{
  "Sid": "Enable IAM User Permissions",
  "Effect": "Allow",
  "Principal": {"AWS": "arn:aws:iam::123456789012:root"},
  "Action": "kms:*",
  "Resource": "*"
}

You can also add the IAM principal in the key policy to allow the CreateGrant API action.

If your AWS KMS key and IAM principal are in the same account, then you can explicitly include your principal in your key policy. You don't need IAM permissions to access the AWS KMS key.

To grant explicit permissions through the key policy, add the following statement to your key policy:

{
  "Principal": {
    "AWS": "arn:aws:iam::123456789012:role/Role_Name"
  },
  "Effect": "Allow",
  "Action": [
    "kms:CreateGrant"
  ],
  "Resource": [
    "*"
  ],
  "Condition": {
    "Bool": {
      "kms:GrantIsForAWSResource": true
    }
  }
}

It's a best practice to grant least privilege for the permissions required to perform a task.

If you use Amazon EC2 Auto Scaling groups to create your instances, then see Required AWS KMS key policy for use with encrypted volumes.

Related information

Amazon EBS encryption

Instance terminates immediately