Windows guest VMs randomly BSOD on AMD EPYC OpenShift Virtualization hosts

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux 9.x, 10.x (KVM)
  • Red Hat OpenShift Virtualization (CNV) 4.x
  • AMD EPYC processors (observed on EPYC 9375F)
  • Microsoft Windows

Issue

Windows guest VMs crash with Blue Screen of Death (BSOD) intermittently and randomly after migration from VMware to OpenShift Virtualization on AMD EPYC hosts. The stop code varies between crashes and between VMs — there is no single consistent bugcheck:

  • 0x3B SYSTEM_SERVICE_EXCEPTION
  • 0xA IRQL_NOT_LESS_OR_EQUAL
  • 0x19 BAD_POOL_HEADER
  • 0x4E PFN_LIST_CORRUPT

Resolution

Solution

Upgrade the hypervisors to the following kernels or higher, depending on the release:

RHEL 9.6: kernel-5.14.0-570.137.1.el9_6
RHEL 9.8: kernel-5.14.0-687.39.1.el9_8
RHEL 10.2: kernel-6.12.0-211.47.1.el10_2

Note: for OpenShift this requires an update of the platform, not just the OpenShift Virtualization operator.

Workarounds on older versions

Disable the tlbflush Hyper-V enlightenment on the affected VM:

In OpenShift Virtualization:

oc patch vm <vm-name> -n <namespace> --type merge -p '{"spec":{"template":{"spec":{"domain":{"features":{"hyperv":{"tlbflush":{"enabled":false}}}}}}}}'

In RHEL KVM:

$ virsh edit <VM NAME>

Change this to off

<features>
  <hyperv>
    <tlbflush state='off'/>
  </hyperv>
</features>

Save and exit.
  • The change only takes effect on the next full VM stop/start — a guest-level reboot or live migration is not sufficient.
  • Apply this only to VMs that are actively crashing, not preventively across the fleet — disabling tlbflush carries a performance penalty (less efficient TLB shootdown), so it should be scoped to the affected VM(s) until an engineering fix is available.
  • If a VM continues to crash after tlbflush is disabled and the VM has been stopped/started, collect a fresh dump — the tlbflush issue is ruled out and the crash should be investigated independently (see Diagnostic Steps).

Root Cause

This is consistent with a known KVM/QEMU issue where the hv-tlbflush Hyper-V enlightenment causes Windows guest memory corruption on AMD hosts. Because the corruption target is essentially random guest kernel memory, the resulting BSOD stop code and the driver blamed in the crash stack vary by which code path happens to touch the corrupted structure next — third-party AV/EDR drivers are common culprits in the stack simply because they perform frequent pool allocation/free and virtual memory operations, not because they are defective.

This was tracked in Content from redhat.atlassian.net is not included.RHEL-214433

A related instance of this class of bug was previously fixed on RHEL 8 KVM hosts / RHV 4.4 (see KCS 6657921, tracked in Bugzilla 1868572).

Diagnostic Steps

  1. Confirm the hosts are AMD: cat /proc/cpuinfo | grep -m1 vendor_id on the node.
  2. Configure the Windows guest to produce a Kernel or Complete memory dump instead of the default Small memory dump/minidump, so a useful dump is captured on the next crash:
    • sysdm.cpl → Advanced → Startup and Recovery → Settings → "Write debugging information" → Kernel memory dump (or Complete memory dump).
  3. Review the crash dump stack. If a third-party AV/EDR driver appears at the top of the stack doing pool allocation/free or NtAllocateVirtualMemory/VirtualAlloc, and the bugcheck code is inconsistent across multiple crashes on the same or similar VMs, suspect this issue rather than the security vendor driver.
  4. Apply the tlbflush workaround above and stop/start the VM.
  5. Monitor for recurrence. If the VM remains stable
Product(s)

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.