RHEL 9 guest panic's during boot with following error "Fatal glibc error: CPU does not support x86-64-v2"
Environment
- Red Hat Enterprise Linux 9
- VMWare ESXi or vSphere
- Microsoft Hyper-V
Issue
- RHEL 9 Virtual Machine fails to boot or to install
- RHEL 7 and 8 Virtual Machine works on the same platform
- Fatal glibc error: CPU does not support x86-64-v2
Fatal glibc error: CPU does not support x86-64-v2
[ 12.457752] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00
Resolution
The Virtualization Platform is likely hiding CPU features to enable live migration compatibility between different hypervisor machines with different CPU models, it needs to be raised to a newer minimum set of features that is compatible with x86-64-v2.
How to do this depends on the platform, please see below:
Red Hat Enterprise Linux
The CPU model is defined in the domain XML from libvirt, see [Documentation - 17.6. Optimizing virtual machine CPU performance](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/configuring_and_managing_virtualization/optimizing-virtual-machine-performance-in-rhel_configuring-and-managing-virtualization#optimizing-virtual-machine-cpu-performance_optimizing-virtual-machine-performance-in-rhel)
Red Hat Virtualization
This is the CPU Model in the Cluster settings, see CPU Type in [Documentation - 2.3.2.2. General Cluster Settings Explained](https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4/html/administration_guide/chap-clusters#Cluster_General_Settings_Explained)
OpenShift Virtualization
This is set on the kubevirt-hyperconverged resource as cpuModel, or per Virtual machine on its yaml definition. See the [Documentation - Scheduling virtual machines](https://docs.openshift.com/container-platform/4.11/virt/virtual_machines/advanced_vm_management/virt-schedule-vms.html).
Red Hat OpenStack Platform
This is set by nova, cpu_model. See cpu_mode, cpu_model and cpu_model_extra_flags in the [Documentation - 9.1.24. libvirt](https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.2/html/configuration_reference/nova_4).
VMWare
This is usually a cluster setting called *EVC Mode*, ensure this is set to the newest possible model in the cluster, or disabled. Older models such as Merom and Penryn do not provide the minimum requirements of RHEL9.
Microsoft Hyper-V
This is usually a setting per VM, under compatibility, named *migration to a physical computer with a different processor*, disabling that should expose a newer CPU model.
Note: for other platforms or if you have additional problems please contact the respective vendor for the platform.
Root Cause
- RHEL9 has a higher CPU requirement than older versions, it now requires a CPU compatible with x86-64-v2 instruction set or higher. This instruction set was introduced around Intel Nehalem models in 2009, so its not very new and any modern CPU should provide it.
- The Virtualization platforms often allow configuring a minimum denominator CPU model, which is a subset or equal to all physical hosts actual CPUs. This is in order to hide features from the VMs so that they can live migrate between hosts with different CPU models, so that the VM does not suddenly misses CPU features when migrating. This setting is likely set to a value too low, below the minimum requirements of RHEL9.
Diagnostic Steps
- In the RHEL9 Virtual Machine console, the kernel panics and an error about missing x86-64-v2 CPU is shown:
[ 12.176528] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 12.215121] Run /init as init process
[ 12.243678] with arguments:
[ 12.260603] /init
[ 12.275386] with environment:
[ 12.294003] HOME=/
[ 12.321100] TERM=linux
[ 12.341965] BOOT_IMAGE=(hd0,gpt2)/vmlinuz-5.14.0-70.el9.x86_64
[ 12.393048] earlyprintl=ttyS0,115200
Fatal glibc error: CPU does not support x86-64-v2 <<<-------
[ 12.457752] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00 <<<---
[ 12.458738] CPU: 0 PID: 1 Comm: init Not tainted 5.14.0-70.el9.x86_64 #1
[ 12.458738] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.0 08/31/2018
[ 12.458738] Call Trace:
[ 12.458738] dump_stack_lvl+0x34/0x44
[ 12.458738] panic+0x102/0x2d4
[ 12.458738] do_exit.cold+0x87/0x9f
[ 12.458738] do_group_exit+0x33/0xa0
[ 12.458738] __x64_sys_exit_group+0x14/0x20
[ 12.458738] do_syscall_64+0x3b/0x90
[ 12.458738] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 12.458738] RIP: 0033:0x7f9c7b743191
[ 12.458738] Code: c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa be e7 00 00 00 ba 3c 00 00 00 eb 0d 89 d0 0f 05 48 3d 00 f0 ff ff 77 1c f4 89 f0 0f 05 <48> 3d 00 f0 ff ff 76 e7 f7 d8 89 05 3f 00 01 00 eb dd 0f 1f 44 00
[ 12.458738] RSP: 002b:00007ffe679b6058 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[ 12.458738] RAX: ffffffffffffffda RBX: 00007f9c7b73de30 RCX: 00007f9c7b743191
[ 12.458738] RDX: 000000000000003c RSI: 00000000000000e7 RDI: 000000000000007f
[ 12.458738] RBP: 000055a2cf252040 R08: 00007ffe679b5bc9 R09: 0000000000000000
[ 12.458738] R10: 00000000ffffffff R11: 0000000000000246 R12: 000000000000000d
[ 12.458738] R13: 0000000000000001 R14: 0000000000000001 R15: 0000000000000001
[ 12.458738] Kernel Offset: 0x29c00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 12.458738] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00 ]---
SBR
Product(s)
Components
Category
Tags
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.