Leapp upgrade fails and drops to emergency mode

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux (RHEL) 7
  • Red Hat Enterprise Linux (RHEL) 8
  • Red Hat Enterprise Linux (RHEL) 9

Issue

  • During reboot phase of the Leapp upgrade the system fails to boot and drops to emergency mode.

    Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot
after mounting them and attach it to a bug report.

Resolution

Perform the steps in the Diagnostic Steps section and gather the sosreport and the rdsosreport.txt file. Upload the sosreport and rdsosreport.txt file to a new case or update your existing case.

Root Cause

A system can drop to emergency mode after the reboot phase of the leapp upgrade for a number of reasons. In order to understand why the system is failing to boot, it is important to gather rdsosreport.txt file and sosreport from the affected system.

Diagnostic Steps

  • Collect the /run/initramfs/rdsosreport.txt file that is generated automatically when the system drops to emergency mode after failed leapp upgrade, and a sosreport. Upload the data to your support case.

Gathering rdsosreport.txt in Emergency Mode

Method 1

  1. Check whether you are able to activate the system storage can be activated and mounted.

    # lvm vgchange -ay

  2. Mount storage:
    a. The example below is using /dev/mapper/rhelvg-lv_root for the root. You will need to adjust for your naming scheme.

    # mount /dev/mapper/rhelvg-lv_root /sysroot

    Note: If you were not able to mount root then proceed with step 2b and 2c. If you were able to mount root then proceed with step 3.

    b. If the above commands fail, please provide the output of:

    # lvm lvs
# blkid
# cat /proc/cmdline

    c. If you were not able to mount root and activate your storage then mount /boot storage device and copy the /run/initramfs/rdsosreport.txt file:

    # mount /dev/sda1 /sysroot
# cp /run/initramfs/rdsosreport.txt /sysroot

  3. Collect the rdsosreport.txt: 

    # cp /run/initramfs/rdsosreport.txt /sysroot/root

  4. Reboot the system and boot to the RHEL 7 kernel (for RHEL 7 -> RHEL 8 upgrades) or RHEL 8 kernel (for RHEL 8 -> RHEL 9 upgrades). Once booted, you will need to correct the selinux labels on the rdsosreport.txt file collected:

    If the file was copied to root the run the below command:

    # restorecon -v /rdsosreport.txt

    If the file was copied to /boot then run this command:

    # restorecon -v /boot/rdsosreport.txt

  5. Upload rdsosreport.txt file to to a new case or update your existing case.

Method 2

Upgrades from RHEL 7 to RHEL 8

  1. Use the steps in the How to upload rdsosreport.txt to another system from the dracut emergency shell article to gather rdsosreport.txt file.

  2. Reboot the system into a working kernel

  3. Gather sosreport and leapp logs.

    # tar cvf /tmp/leapp-logs.tar /var/log/leapp/
# sosreport || sos report

For a video overview of different methods, please see:


SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.