How do I troubleshoot a Linux WorkSpace that's in the Unhealthy state?

Short description

WorkSpaces periodically sends a health status request to each WorkSpace to check the health of the WorkSpace. If WorkSpaces doesn't receive a response from the WorkSpace, then the WorkSpace status changes to Unhealthy.

The following issues can cause the status to change to Unhealthy:

• The WorkSpace computer name changed, and you didn't reboot the WorkSpace.
• The WorkSpace is consistently using high CPU.
• The agent or service that responds to WorkSpaces isn't running, or the management interface (ETH0) is turned off.
• An Amazon DCV or PCoIP service isn't running.
• Antivirus software is blocking WorkSpaces components.
• An application on the WorkSpace is blocking the network connection between WorkSpaces and the WorkSpace on the management interface.

Resolution

Use CloudWatch metrics to review your WorkSpaces

To help you determine the cause, review the CPUUsage, MemoryUsage, and Unhealthy WorkSpaces metrics in Amazon CloudWatch.

Reboot the WorkSpace

Reboot the WorkSpace. If a reboot doesn't resolve the issue, then use SSH to connect to the WorkSpace.

Note: By default, SSH is turned off on Ubuntu and Red Hat Enterprise Linux (RHEL) WorkSpaces. To use SSH, turn on SSH for your Ubuntu or RHEL WorkSpace.

If you can't use SSH to connect to your WorkSpace, then proceed to the Restore or rebuild the WorkSpace section.

Check for high CPU

Check whether your Amazon Elastic Compute Cloud (Amazon EC2) Linux instance has high CPU utilization.

Check that the management and customer interfaces are running

To check for active interfaces, run the following command:

sudo ifconfig

To check all available interfaces, run the following command:

sudo ip link show

If an interface isn't running, then run the following command to activate the interface:

sudo ifconfig ethernet-name up

Note: Replace ethernet-name with your Ethernet name.

Confirm that the WorkSpaces services are running and responsive

Use SSH to connect to the WorkSpace. Then, run the commands for your service to check the service's status.

Ubuntu or RHEL WorkSpace:

sudo systemctl status skylight-agent.service 
sudo systemctl status wspdcvhostadapter.service
sudo systemctl status dcvserver.service

Amazon Linux 2 WorkSpace:

sudo systemctl status skylight-agent.service 
sudo systemctl status pcoip.service

If the service is in the Stopped state, then run the commands for your service to start the service.

Ubuntu or RHEL WorkSpace:

sudo systemctl start skylight-agent.service
sudo systemctl start wspdcvhostadapter.service
sudo systemctl start dcvserver.service

Amazon Linux 2 WorkSpace:

sudo systemctl start skylight-agent.service 
sudo systemctl start pcoip.service

If the services are running, then run the command for your service to check status of the service ports.

Skylight:

sudo netstat -tulpn | grep skylight

DCV services that use Ubuntu and RHEL:

sudo netstat -ntpla |grep dcv

PCoIP services that use Amazon Linux 2:

sudo netstat -ntpla |grep pcoip

The status for each service must be LISTEN.

Example:

sudo netstat -ntpla |grep dcv
tcp 0 0 127.0.0.1:8290 0.0.0.0:* LISTEN 1058/wspdcvhostadap
tcp 0 0 127.0.0.1:9999 0.0.0.0:* LISTEN 1058/wspdcvhostadap
tcp 0 0 198.19.129.139:8220 0.0.0.0:* LISTEN 1073/dcvserver

Verify your WorkSpaces configuration

Verify that endpoint protection software, such as antivirus or anti-malware software, allows the required WorkSpaces service components. Also, verify that an application or VPN isn't blocking your management adapter. Then, check your WorkSpace connectivity.

Use the /var/lib/skylight/tls.cert file path to verify that the Skylight certificate is in the Linux certificate store.

Note: This file location is same for all Linux distributions.

Verify firewall rules

The firewall must allow listed traffic on the management network interface. Also, verify that the operating system (OS) firewall or third-party firewall has rules to allow the required ports.

Restore or rebuild the WorkSpace

If you can't use SSH to connect to the WorkSpace, then restore the WorkSpace to the latest snapshot. If the WorkSpace is still unhealthy, then rebuild the WorkSpace.

To restore or rebuild the WorkSpace, it's a best practice to use the AWS Systems Manager AWSSupport-RecoverWorkSpace runbook.

Important: When you restore or rebuild a WorkSpace, data loss can occur. The WorkSpace is restored from the last available snapshot that's up to 12 hours old. Rebuild recreates the user volume from the most recent snapshot and the WorkSpace from the image of the bundle that you created the WorkSpace from. Applications that you installed or system settings that you changed after you created the WorkSpace are lost.

Before you run the automation, make sure that your AWS Identity and Access Management (IAM) user or role has the required permissions. For more information, see the Required IAM permissions section of AWSSupport-RecoverWorkSpace.

To run the runbook, complete the following steps:

1. Open the AWSSupport-RecoverWorkSpace runbook.
2. Choose Execute automation.
3. For the input parameters, enter the following values:
   (Optional) For AutomationAssumeRole, enter the ARN of the IAM role that allows the automation to perform actions. If you don't specify a role, then the automation uses the permissions of the user that starts the runbook.
   For Acknowledge, enter Yes to acknowledge that the Restore and Rebuild actions recover the WorkSpace from the most recent snapshot.
   For Reboot, Rebuild, or Restore, choose Yes for your preferred option.
   For WorkspaceId, enter the ID of the WorkSpace that you want to recover.
4. Choose Execute.
   Note: For a list of the steps that the runbook performs, see the Document steps section of AWSSupport-RecoverWorkSpace.
5. Check the status of your Workspace in the Output section of the runbook.

You can also use the AWS Command Line Interface (AWS CLI) to reboot, restore, or rebuild the WorkSpace.

Note: If you receive errors when you run AWS CLI commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

If none of the preceding troubleshooting steps resolve your issue, then collect the client-side logs and open an AWS Support case.

Related information

IP address and port requirements for WorkSpaces Personal

Turn on self-service WorkSpace management capabilities for your users in WorkSpaces Personal

How do I troubleshoot a Windows WorkSpace that's in the Unhealthy state?