How do I troubleshoot Windows patch installation failures in Systems Manager?

Short description

The following issues cause Windows patch installation failures in Patch Manager, a capability of AWS Systems Manager:

• Network connectivity issues that prevent access to the Windows Update service or the Windows Server Update Services (WSUS) server.
• Group Policy configurations that enforce unreachable WSUS servers.
• System file that's corrupted on the Amazon Elastic Compute Cloud (Amazon EC2) Windows instance.

If you have these issues, then Run Command, a capability of AWS Systems Manager, returns a "0x80240438" or "0x800f0985" Windows Knowledge Base (KB) error code and a Failed status.

Resolution

Prerequisite: Verify that your instance meets the Patch Manager prerequisites, and then create a backup of your instance. If you experience issues when you perform the following troubleshooting steps, then restore your instance from the backup that you created.

Resolve network connectivity issues that prevent patch downloads

If your instance can't reach Windows Update or your WSUS server, then you might receive "The remote name could not be resolved" error message on your patch baseline snapshots. Or, if your instance can't reach the Windows Update server, then multiple Windows KB updates fail when you install them.

To resolve, complete the following steps:

1. Open the AWS Systems Manager console.

2. In the navigation pane, choose Run Command.

3. Review the failed SSM Run Command execution to identify the specific error message.

4. Use Session Manager, a capability of AWS Systems Manager, to connect to your instance.

5. To test your outbound network connection, run the following PowerShell command:

   Test-NetConnection patch-baseline-snapshot-example-region.s3.example-region.amazonaws.com -port 443

   Note: Replace example-region with your AWS Region code. If your Region uses a unique suffix, then replace the first instance of example-region with your Region code and its suffix. For a list of Regions and their suffixes, see Buckets containing AWS managed patch baseline snapshots.
   Example command with the Region as South Africa (Cape Town):

   Test-NetConnection patch-baseline-snapshot-af-south-1-tbxdb5b9.s3.af-south-1.amazonaws.com -port 443

6. In the output, verify that you have outbound network connectivity on HTTPS port 443.

7. To check your DNS resolution to the necessary virtual private cloud (VPC) endpoints, run the following command:

   nslookup patch-baseline-snapshot-example-region.s3.example-region.amazonaws.com

   Note: Replace example-region with your AWS Region code. If your Region uses a unique suffix, then replace the first instance of example-region with your Region code and its suffix.

8. In the output, verify that your instance can resolve the endpoint that Patch Manager uses to download patch baseline snapshots.

9. To review the status of your Windows Firewall profiles, run the following PowerShell command:

   Get-NetFirewallProfile | Format-Table Name, Enabled

10. In the output, verify that you have outbound network connectivity on HTTPS port 443.

11. To display your system's Windows HTTP Services (Win HTTP) proxy settings, run the following command:

    netsh winhttp show proxy

12. In the output, check whether you configured a system-level proxy.
    Note: If you use a proxy, then make sure that you correctly configured AWS Systems Manager Agent (SSM Agent) to use a proxy.

13. If your instance is in a private subnet, then check that your instance uses a NAT gateway to reach the internet.

14. If your instance is in a public subnet, then check that your instance's route table has a route to the internet. Check that your Amazon Virtual Private Cloud (Amazon VPC) security groups allow outbound connections on port 443. Then, verify that your network access control lists (network ACLs) allow inbound and outbound connections on port 443.

15. If your instance uses Transit Gateway, then check that your Transit Gateway allows inbound and outbound traffic from your private subnets.

Resolve unreachable WSUS server issues

If your instance's Group Policy settings enforce Windows updates through WSUS, then you get errors that indicate that the WSUS server is unreachable or incorrectly configured. Patch baseline operations fail with "Exception from HRESULT" error messages or "0x80240438" error messages.For Windows Update, logs show errors that communicate with WSUS endpoints.

To resolve, complete the following steps:

1. Check that you can install a patch on an instance that you didn't join to a domain.
   Note: If Patch Manager doesn't install the patch, then check that your instance has internet connectivity.

2. On the instance that has the error, run the following command to generate Windows Update logs:

   Get-WindowsUpdateLog

3. Review the logs for entries that have the following error messages:
   "WS error: There was an error communicating with the endpoint"
   "WS error: The given proxy cannot be reached"

4. Navigate to the HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\Windows\WindowsUpdate\AU folder.

5. Check your UseWUServer value.
   If your UseWUServer value is 1, then check that your instance can connect with the WSUS server that you specify in your Group Policy. Also, check that you correctly configured the WSUS server. Then, contact your administrator to confirm that you correctly configured your update source
   If your UseWUServer value is 0, then update your Group Policy to allow instances to use Windows Update.

Resolve system file corruption on your instance

Corrupt systems files prevent Windows Update patch installations, even when network connectivity and Group Policy settings are correct. If you have corrupt system files that cause your Windows patch installation failures, then you might receive the following errors:

• Patching operations fail when Run Command runs successfully.
• Component-Based Servicing (CBS) logs show Component Servicing Infrastructure (CSI) Payload Corruption errors.
• The RestoreHealth command in the Deployment Image Servicing and Management (DISM) repair tool fails. For more information, see Use the System File Checker tool to repair missing or corrupted system files on the Microsoft website.
• Individual KB updates fail and return "-2145124475" or "0x800f0985" error messages.

To check for corrupted system files, complete the following steps:

1. Review your CBS logs at %systemroot%\Logs\CBS.
2. In your logs, look for CSI Payload Corrupt entries.
3. Run the System File Checker command to identify and repair corrupt system files:

   sfc /scannow

   Note: If the System File Checker command doesn't repair the corrupt system files that it finds, then rerun the command. If the command can't repair your system files after three tries, then proceed to the next step.
4. Run the following CheckHealth command to check the Component Store of the Windows image for corrupt files:

   DISM /Online /Cleanup-Image /CheckHealth

   Note: For information, see /Cleanup-Image on the Microsoft website.
5. If the CheckHealth command detects corruption, then run the following ScanHealth command to check that the image is repairable:
   DISM /Online /Cleanup-Image /ScanHealth
   If the image is repairable, run the following RestoreHealth command:
   DISM /Online /Cleanup-Image /RestoreHealth
   Note: If the RestoreHealth command successfully runs, then proceed to step 10. If the command fails, then use another functional Windows Server instance as the repair source.
   If the image is non-repairable, then discard the image and launch a new instance from a clean Amazon Machine Image (AMI). Restore your workload from a backup or snapshot taken before the corruption occurred. For more information, see Repair a Windows Image on the Microsoft website.
6. To verify that you have administrative access to a server, run the following command:

   DISM.exe /Online /Cleanup-Image /RestoreHealth /Source:\\example-server-name\c$\winsxs /LimitAccess

   Note: Replace example-server-name with the hostname of a functional Windows Server instance that runs the same version of the operating system (OS). If you receive an "Access denied" error, then run the following command to map the network drive with credentials:

   net use \\example-server-name\c$ /user:domain\username  
   

   Note: Replace example-server-name with the hostname of a functional Windows Server instance that runs the same OS version.
7. Check that your account has administrative permissions on the source server and that you activated network file sharing.
8. Check that the instances can communicate over Server Message Block (SMB) ports TCP 445 and 139.
9. Run the following command to map the network drive with credentials:

   net use \\example-server-name\c$ /user:domain\username

   Note: Replace example-server-name with the hostname of a functional Windows Server instance that runs the same OS version.
10. Download the necessary patch, and then install it on the instance. To download your patch, see the Microsoft Update Catalog on the Microsoft website.
11. Reset your Windows Update components. For instructions, see Reset Windows Update components manually on the Microsoft website.
12. Reboot your instance.
13. Run the following Get-Service PowerShell command to verify that SSM Agent is active:

    Get-Service AmazonSSMAgent

    Note: For information, see Get-Service on the Microsoft website
14. If SSM Agent isn't active, then run the following Start-Service PowerShell command to start it:

    Start-Service AmazonSSMAgent

    Note: For more information, see Start-Service on the Microsoft website.

Complete additional troubleshooting steps

If your issue persists, then review your logs and troubleshoot:

• For Patch Baseline Operations logs, review %PROGRAMDATA%\Amazon\PatchBaselineOperations\Logs.
• Review your EC2Rescue logs.
• To review your Windows Update logs, run the following PowerShell command:

  Get-WindowsUpdateLog

  Note: For information, see Get-WindowsUpdateLog on the Microsoft Website.
• For Component-Based Servicing logs, review %systemroot%\Logs\CBS.

If your component store or registry is corrupted, then perform an in-place upgrade to the same version of the OS that you use.

Related information

How Patch Manager operations work

Troubleshooting Patch Manager