How to capture a vmcore on a hung KVM guest via NMI

Environment

• Red Hat Enterprise Linux
• KVM

Issue

• How do I capture a vmcore from a hung KVM guest?
• How to send NMI signal to a KVMguest?

Prerequisites

• Kdump should be running and tested beforehand. Follow this article to configure and test the Kdump utility:
  How to troubleshoot kernel crashes, hangs, or reboots with kdump on Red Hat Enterprise Linux

• Alternatively, This content is not included.Kdump Helper may be considered for Kdump configuration.

• NMI panic tunables need to be enabled.

Resolution

The virsh inject-nmi domain injects a non-maskable interrupt (NMI) message to the guest virtual machine. This is useful when the guest VM is in a hung state, as it allows for manual collection of a vmcore file, which can be crucial for debugging and analyzing the cause of the system hang.

Note: This process is different from collecting a memory dump (# virsh dump) from a KVM guest.

• From the KVM host machine, run the following command to list the domain name:

# virsh list --all

Example:

    # virsh list --all
     Id   Name                 State
    -------------------------------------
     1    guest-VM-1           running

• From the KVM host machine, run the following command to inject/run the NMI command on the KVM guest using the domain name captured from above step in-place of <DOMAIN NAME>:

# virsh inject-nmi <DOMAIN NAME>

Example:

    # virsh inject-nmi guest-VM-1

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.