How do I troubleshoot issues with AWS Systems Manager Session Manager?

Resolution

If a session fails because your Amazon Elastic Compute Cloud (Amazon EC2) instance isn't available as a managed instance, then troubleshoot your managed instance availability.

If a session fails and your Amazon EC2 instance is available as a managed instance, then troubleshoot Session Manager to resolve the following issues:

• Session Manager doesn't have permission to start a session.
• Session Manager doesn't have permission to change session preferences.
• A managed node isn't available or isn't configured for Session Manager.
• Session Manager plugins aren't added to the command line path (Windows).
• The system sends a TargetNotConnected error.
• Session Manager shows a blank screen when you start a session.

If a session fails and shows one of the following error messages, then apply the appropriate troubleshooting guidance.

"Your session has been terminated for the following reasons: ----------ERROR------- Encountered error while initiating handshake. Fetching data key failed: Unable to retrieve data key, Error when decrypting data key AccessDeniedException: The ciphertext refers to a AWS KMS key that does not exist, does not exist in this region, or you are not allowed to access. status code: 400, request id: nnnnnnnnnnnn"

You receive this error when the users and EC2 instances in your account don't have the required AWS Key Management Service (AWS KMS) key permissions. To resolve this error, turn on AWS KMS encryption for your session data, and then complete the following steps:

1. Grant the required AWS KMS key permissions to the users who start sessions and the instances that the sessions connect to.
2. Configure AWS Identity and Access Management (IAM) to provide the users and instances with permissions to use the AWS KMS key with Session Manager:
   To add AWS KMS key permissions for users, see Sample IAM policies for Session Manager.
   To add AWS KMS key permissions for instances, see Verify or add instance permissions for Session Manager.
   For Default Host Management Configuration, add a policy to an IAM role that provides AWS KMS key permissions.

Note: In AWS Systems Manager Agent (SSM Agent) version 3.2.582.0 and newer, Default Host Management Configuration automatically manages EC2 instances without an IAM instance profile. The instances must use Instance Metadata Service Version 2 (IMDSv2).

"Error - Fleet Manager is unable to start the session because the WebSocket connection closed unexpectedly during the handshake. Verify that your instance profile has sufficient Sessions Manager and AWS KMS permissions. For a more detailed message, visit the Session Manager console"

You might get this error when the target instance attached instance profile role is missing the following permission. To use AWS Systems Manager with AWS KMS, the kms:Decrypt permission is required to allow customer key encryption and decryption for session data.

{ 
    "Effect":   "Allow", 
    "Action":  [ 
          "kms:Decrypt" 
    ], 
    "Resource": "key-name" 
}

To resolve this error, update your permissions of the instance profile role attached to your instance. For Session Manager permissions example, see Add Session Manager permissions to an existing IAM role.

"Your session has been terminated for the following reasons: Couldn't start the session because we are unable to validate encryption on Amazon S3 bucket. Error: AccessDenied: Access Denied status code: 403"

You receive this error when you choose Allow only encrypted S3 buckets for S3 logging in your Session Manager preferences. To resolve the error, complete the following steps:

1. Open the AWS Systems Manager console.
2. Choose Session Manager, Preferences, and then choose Edit.
3. Under S3 logging, clear Allow only encrypted S3 buckets, and then save your changes.
   For more information, see Logging session data using Amazon S3 (console).
4. For instances managed with an IAM instance profile, add a policy to the instance profile that provides permissions to upload encrypted logs to Amazon S3. For instructions, see Creating an IAM role with permissions for Session Manager and Amazon S3 and Amazon CloudWatch Logs (console).
5. For instances managed with Default Host Management Configuration, add a policy to the IAM role that provides permissions to upload encrypted logs to Amazon S3. For more information, see Creating an IAM role with permissions for Session Manager and Amazon S3 and CloudWatch Logs (console).

"Your session has been terminated for the following reasons: We couldn't start the session because encryption is not set up on the selected CloudWatch Logs log group. Either encrypt the log group or choose an option to enable logging without encryption"

You receive this error when you choose Allow only encrypted CloudWatch log groups for CloudWatch logging in your Session Manager preferences. Complete the following steps to resolve the error:

1. Open the AWS Systems Manager console.
2. Choose Session Manager, Preferences, and then choose Edit.
3. Under CloudWatch logging, clear Allow only encrypted CloudWatch log groups, and then save your changes.
   For more information, see Logging session data using Amazon CloudWatch Logs (console).
4. For instances managed with IAM instance profile, add a policy to the instance profile that provides permissions to upload encrypted logs to Amazon CloudWatch. For instructions, see Creating an IAM role with permissions for Session Manager and Amazon S3 and CloudWatch Logs (console).
5. For instances managed with Default Host Management Configuration, add a policy to the IAM role that provides permissions to upload encrypted logs to CloudWatch. For instructions, see Creating an IAM role with permissions for Session Manager and Amazon S3 and CloudWatch Logs (console).

"Your session has been terminated for the following reasons: ----------ERROR------- Unable to start command: Failed to create user ssm-user: Instance is running active directory domain controller service. Disable the service to continue to use session manager"

You might get this error when you use AWS Systems Manager for Windows Server. The cause for this error depends on the version of SSM agent running on the instance.

• In SSM agent version 2.3.612.0 and newer, the ssm-user account isn't automatically created on Windows Server machines that are used as domain controllers. The user account with the name ssm-user must be created manually and set with required permissions.
• To identify the SSM agent version, see Checking the SSM Agent version number. Also review the SSM Agent version release notes on the GitHub website.
• To subscribe to SSM agent notification, see Subscribing to SSM Agent notifications.
• To configure SSM Agent to automatically update, see Automating updates to SSM Agent.

Related information

How do I attach or replace an instance profile on an Amazon EC2 instance?

Enabling and disabling session logging

Setting up Session Manager