- Newest
- Most votes
- Most comments
By running these checks, you have ruled out ENA driver misconfiguration and forced synthetic timers.
However, your bcdedit output reveals a fundamental architectural divergence between your two deployments:
- WS2022 is booting via
\Windows\system32\winload.exe(Legacy BIOS). - WS2025 is booting via
\EFI\Microsoft\Boot\bootmgfw.efiandwinload.efi(UEFI).
My understanding is, that this is a critical distinction. When a Windows instance boots in UEFI mode on the AWS Nitro system, the Hardware Abstraction Layer (HAL) interacts differently with the hypervisor compared to Legacy BIOS. Specifically, ACPI table parsing, timer routing (e.g., HPET, RTC, synthetic timers), and the interrupt controller initialization (APIC/x2APIC) are handled via entirely different execution paths.
The baseline interrupt storm (~1,500 interrupts/sec) you are observing is almost certainly a symptom of how the WS2025 kernel/HAL is mapping or polling hardware interrupts in this specific UEFI environment on the underlying AMD EPYC (t3a) architecture.
To answer your question directly: Yes, a Windows Performance Recorder (WPR) trace is exactly the correct next step. As far as I know, it's the only way to move from hypothesis to absolute certainty by proving exactly which driver or system routine is triggering the interrupts.
How to proceed with the WPR Trace
1. Capture the Trace Open an elevated command prompt on the idle WS2025 instance and run:
wpr -start cpu -start Interrupt
Let the instance sit completely idle for 30–60 seconds, then stop and save the trace:
wpr -stop C:\high_interrupts.etl
2. Analyze the Trace (Locally)
If you open high_interrupts.etl using Windows Performance Analyzer (WPA), expand the Computation graph and view DPC/ISR or CPU Usage (Sampled). Group the columns by Module Name and Function Name. The module consuming the CPU cycles (e.g., hal.dll, ntoskrnl.exe, ACPI.sys, or a specific Nitro driver) will immediately float to the top.
3. Escalation and Security Warning
Please do not upload the raw .etl file here on re:Post. WPR traces contain deep system memory pools, file paths, and potentially sensitive background data.
Because this is a low-level OS/Hypervisor interaction issue, you need to open an AWS Technical Support case. Provide them with:
- Your reproduction steps (WS2025/UEFI vs. WS2022/Legacy on
t3a.micro). - The
.etltrace file.
This will provide the EC2 Windows Engineering team with the exact stack trace they need to either calibrate the WS2025 AMI or patch the respective Nitro driver.
If you do open the trace in WPA, please feel free to share just the name of the faulting module here in the thread! It would be highly valuable for the community to know exactly what is causing the spike.
see also:
Evidence for the Legacy BIOS vs. UEFI difference (winload.exe vs. winload.efi) AWS clearly documents that EC2 instances (especially on the Nitro system) boot in either Legacy BIOS mode or UEFI mode depending on the AMI. Microsoft confirms that the Windows bootloader path explicitly reveals which mode the OS is running in:
Microsoft Docs: BCD System Store Settings for UEFI (Confirms that \Windows\system32\winload.efi is used exclusively for UEFI systems, whereas legacy systems use winload.exe.) -> https://learn.microsoft.com/en-us/windows-hardware/manufacture/desktop/bcd-system-store-settings-for-uefi
AWS Docs: Boot modes for Amazon EC2 instances (Explains how the Nitro system provides different underlying firmware interfaces for UEFI and Legacy BIOS.) -> https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ami-boot.html
Evidence for altered Interrupt & Timer routing under UEFI Under UEFI, the way the operating system's Hardware Abstraction Layer (HAL) interacts with hardware timers (such as HPET or invariant TSC) and the Advanced Configuration and Power Interface (ACPI) fundamentally changes. This handoff between the hypervisor firmware and the OS dictates how interrupts are routed (e.g., APIC vs. x2APIC), which directly impacts virtualized CPU overhead:
Microsoft Docs: Analyze High CPU Usage with WPA *(Documents the exact workflow for recording interrupt storms using wpr -start cpu and how to visualize DPC/ISR spikes down to the exact .sys driver level in the Windows Performance Analyzer). * -> https://learn.microsoft.com/en-us/windows-hardware/test/wpt/cpu-analysis
I think the generic SSM runbooks mentioned above won't help here. A high % Interrupt Time (~11%) on a clean, idle OS indicates a low-level issue with kernel timer handling or hypervisor clock synchronization in Windows Server 2025 on Nitro (t3a / AMD EPYC).
To isolate the root cause, test these specific configurations on a fresh test instance:
- Disable Dynamic Ticks
Windows Server 2025 uses dynamic ticking to adjust timer resolution on the fly. On virtualized hardware, miscommunication between guest timer interrupts and the hypervisor frequently causes an interrupt storm. bcdedit /set disabledynamictick yes
(Requires a system reboot).
- Verify ENA Interrupt Moderation
Check if the Amazon Elastic Network Adapter driver in WS2025 has interrupt moderation enabled by default: Get-NetAdapterAdvancedProperty -Name "Ethernet" | Where-Object { $_.DisplayName -like "Interrupt" }
If moderation is disabled or misconfigured, small background packets can saturate a vCPU with hardware interrupts.
- Compare BCD Clock Settings
Run bcdedit /enum on both WS2022 and WS2025. Verify if useplatformclock or useplatformtick is enabled on the WS2025 AMI, which forces the OS to use synthetic clock sources instead of invariant TSC, causing higher overhead.
If disabling dynamic ticks drops % Interrupt Time back to WS2022 baseline levels, attach your before/after Get-Counter logs to an AWS Support case so this can be escalated to the EC2 Windows Engineering team for AMI timer calibration.
See also:
Thank you, I opened a case.
I also found out that this does NOT happen with non-burstable instance types like c7a!
WPA analysis attributes almost all ISR CPU time to ntoskrnl!HalpVpptAcknowledgeInterrupt, with the call stack dominated by:
ntoskrnl!KiIdleLoop → ntoskrnl!PoIdle → ntoskrnl!PpmIdleExecuteTransition → ntoskrnl!HalpVpptAcknowledgeInterrupt
Very little ISR time is attributed to ENA, AWSNVMe or Storport.
Based on your detailed analysis, you're experiencing high interrupt processing on Windows Server 2025 instances that doesn't occur on Windows Server 2022. The approximately 10% CPU usage from interrupt processing on idle instances is concerning, especially for burstable instances like t3a.micro that can't accumulate CPU credits when consistently above their baseline.
To troubleshoot and potentially resolve this issue:
Immediate Troubleshooting Steps:
-
Run the AWSSupport-TroubleshootWindowsPerformance runbook through AWS Systems Manager. This can automatically identify processes causing high CPU usage and may provide additional insights into the interrupt processing behavior.
-
Check for driver compatibility issues by running the AWSSupport-UpgradeWindowsAWSDrivers runbook. While your driver versions appear current, Windows Server 2025 is relatively new, and there may be compatibility improvements or updates available specifically for this OS version.
-
Monitor with CloudWatch by setting up detailed monitoring and creating alarms for CPUUtilization. Use the procstat plugin to get more granular process-level data that might reveal patterns not visible in standard metrics.
Potential Root Causes:
Given that Windows Server 2025 is a newer operating system, there may be:
- Compatibility issues between the OS kernel's interrupt handling and the current AWS drivers (ENA, NVMe)
- Changes in how Windows Server 2025 processes hardware interrupts compared to 2022
- Differences in default system services or background tasks that generate more interrupt activity
Recommended Actions:
- Ensure all instance drivers are fully updated and compatible with Windows Server 2025
- Consider temporarily using a larger instance type (beyond t3a.micro) to determine if the issue scales proportionally or if it's a fixed overhead
- Review your AWS Support plan options, as this appears to be a systemic compatibility issue that may require deeper investigation by AWS engineering teams
- Document your findings and add correspondence through the AWS Support Center at https://console.aws.amazon.com/support/ with your detailed performance counter data
Since this is reproducible across multiple deployments and availability zones, it's unlikely to be a host-specific issue and more likely represents a broader compatibility consideration with Windows Server 2025 on AWS infrastructure.
Sources
Troubleshoot high CPU usage in my EC2 Windows instances | AWS re:Post
Troubleshoot EC2 Windows Nitro instance status check failures | AWS re:Post
answered 24 days ago
Thank you @Florian Turnwald for your super quick and helpful answer!
Output for (2) on Server 2025:
Empty. Output without the filter:
Name DisplayName DisplayValue RegistryKeyword RegistryValue
---- ----------- ------------ --------------- -------------
Ethernet IPv4 Checksum offload Rx & Tx Enabled *IPChecksumO... {3}
Ethernet Jumbo Packet Disabled *JumboPacket {1514}
Ethernet Large Send Offload V1 (IPv4) Enabled *LsoV1IPv4 {1}
Ethernet Large Send Offload V2 (IPv4) Enabled *LsoV2IPv4 {1}
Ethernet Large Send Offload V2 (IPv6) Enabled *LsoV2IPv6 {1}
Ethernet Maximum Number of RSS Queues 32 *NumRssQueues {32}
Ethernet Priority and Vlan Support Both *PriorityVLA... {3}
Ethernet Receive Buffers 1024 *ReceiveBuffers {1024}
Ethernet Receive Side Scaling Enabled *RSS {1}
Ethernet TCP Checksum Offload (IPv4) Rx & Tx Enabled *TCPChecksum... {3}
Ethernet TCP Checksum Offload (IPv6) Rx & Tx Enabled *TCPChecksum... {3}
Ethernet Transmit Buffers 1024 *TransmitBuf... {1024}
Ethernet UDP Checksum Offload (IPv4) Rx & Tx Enabled *UDPChecksum... {3}
Ethernet UDP Checksum Offload (IPv6) Rx & Tx Enabled *UDPChecksum... {3}
Ethernet LLQ Header Size Policy Recommended LlqPolicy {0}
Ethernet Metrics Refresh Interval 1 MetricsRefre... {1}
Ethernet MAC Address -- NetworkAddress {--}
Ethernet VLAN ID 0 VlanId {0}
Same on Server 2022.
Output for (3)
On Server 2025:
Windows Boot Manager
--------------------
identifier {bootmgr}
device partition=\Device\HarddiskVolume2
path \EFI\Microsoft\Boot\bootmgfw.efi
description Windows Boot Manager
locale en-US
inherit {globalsettings}
bootshutdowndisabled Yes
default {current}
resumeobject {06abedb4-730f-11f0-bb11-0afff7ff4a31}
displayorder {current}
toolsdisplayorder {memdiag}
timeout 30
Windows Boot Loader
-------------------
identifier {current}
device partition=C:
path \Windows\system32\winload.efi
description Windows Server
locale en-US
inherit {bootloadersettings}
recoverysequence {5e607f64-734b-11f0-aaf4-0affe412849f}
displaymessageoverride Recovery
recoveryenabled Yes
isolatedcontext Yes
allowedinmemorysettings 0x15000075
osdevice partition=C:
systemroot \Windows
resumeobject {06abedb4-730f-11f0-bb11-0afff7ff4a31}
nx OptOut
bootstatuspolicy IgnoreAllFailures
useplatformclock or useplatformtick not mentioned!
Neither on Server 2022, although the output is slighty different:
Windows Boot Manager
--------------------
identifier {bootmgr}
device partition=C:
description Windows Boot Manager
locale en-US
inherit {globalsettings}
bootshutdowndisabled Yes
default {current}
resumeobject {4426d739-00bc-11ec-a967-0e221fdbf186}
displayorder {current}
toolsdisplayorder {memdiag}
timeout 30
Windows Boot Loader
-------------------
identifier {current}
device partition=C:
path \Windows\system32\winload.exe
description Windows Server
locale en-US
inherit {bootloadersettings}
recoverysequence {abb264d7-85c0-11ef-bb37-0afff4384355}
displaymessageoverride Recovery
recoveryenabled Yes
allowedinmemorysettings 0x15000075
osdevice partition=C:
systemroot \Windows
resumeobject {4426d739-00bc-11ec-a967-0e221fdbf186}
nx OptOut
bootstatuspolicy IgnoreAllFailures
Disable Timer Ticks
I ran the suggested command and rebooted. Unfortunately, it didn't change anything except the output of bcdedit /enum which now includes
disabledynamictick Yes
Any ideas? Should I provide you with a WPR trace (CPU profile)?
Relevant content
asked a year ago
asked 2 years ago
asked 2 years ago

I guess it makes sense to share the relevant part of the response I got:
I fully understand that problems like this might be difficult to resolve, but it's very unfortunate that this isn't documented anywhere publicly, and as a customer you have to burn hours to analyze the problem and need paid support to get confirmation that it is a known AWS problem...
Once I can confirm that the problem has been resolved, I will add an update here.