How to use VMWare ESX command line to force NMI panic on RHEL guest O/S

Environment

• VMWare ESX 3 or later host
• Red Hat Enterprise Linux VM guest

Issue

• We have a server that stops responding but nothing is recorded in messages. When this occurs, our only option has been to reboot the server.

Resolution

• The VMWare ESX server has a command line mode where this can be done.

• Set the following for the RHEL 5 VM guest:

kernel.panic_on_unrecovered_nmi = 1
kernel.unknown_nmi_panic = 1

• Set the following for the RHEL 6 and later VM guest:

kernel.panic_on_unrecovered_nmi = 1
kernel.unknown_nmi_panic = 1
kernel.panic_on_io_nmi = 1

• Make sure kdump is configured and started:

service kdump start

or the following operation on RHEL 7 and above:

systemctl start  kdump

• When the guest hangs, from the VMWare ESX host issue the following command:

In order to find the <world id> on the esxi console, you can run based on version:
ESXi: /sbin/vmdumper -l
ESX: /usr/lib/vmware/bin/vmdumper -l

The the following command can be executed based on version:
ESXi: /sbin/vmdumper <world_id> nmi
ESX:  /usr/lib/vmware/bin/vmdumper <world_id> nmi

• Where <world_id> is the guest instance. This will cause an NMI on the RHEL guest and a panic will be forced. For further information regarding how to determine the '<world_id>' information, please reference the following VMWare provided documentation regarding this process:

• If this process fails to generate a vmcore, then you can use the following process to Content from kb.vmware.com is not included.generate a vmcore.
  In the above guide, just choose the resolution based on the version that applies to you.

- ESXi 5.x and vSphere Client 
- ESXi 5.x and WebClient

• Any configuration changes are required for ESXi host in order to run "vmdumer". Just enable SSH option on ESXi.

• To do this:
  Access ESXi console > Press [F2] (Customize System/View Logs) > [System Customization] > [Troubleshooting Options] > [Enable SSH]

Content from kb.vmware.com is not included.Generating a Windows core dump to troubleshoot unresponsive virtual machines on ESX/ESXi (1009187)

Warning: The above link is to a source that are not authored by Red Hat directly. As such, Red Hat support cannot verify its accuracy and content.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.