How to start Red Hat CoreOS (RHCOS) in emergency mode in OpenShift 4

Solution Verified - Updated

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4
  • Red Hat Enterprise Linux CoreOS (RHCOS)
    • 4

Issue

  • Due to a network miss-configuration the machine may not be reachable through SSH.
  • There is no default password set to log in on the console.
  • How to use rd.break to access RHCOS OpenShift Container Platform 4 nodes.
  • How to access emergency mode in RHCOS OpenShift Container Platform 4.
  • How to use rd.break to access RHCOS OpenShift Container Platform 4 nodes to repair filesystems.

Resolution

Refer to accessing the RHCOS Console to connect to a running console (only possible in some platforms).

Starting current RHCOS in emergency mode

On RHCOS 4.7 (OCP 4.7) or later, can follow the instructions below to boot the node in emergency mode:

  • Intercept the GRUB menu.
  • Edit the entry by using the e key.
  • By default two console parameters are provided in the kernel command line, only keep the one you need. For example remove the entry for: console=ttyS0,115200n8.
  • Add single to the kernel command line.
  • Press Ctrl+X to resume booting.

Starting dracut (initramfs) shell


Sometimes, it may just needed to open the `dracut` shell to do some actions in the initramfs, such as running `xfs_repair` to repair the main filesystem:
  • Intercept the GRUB menu.
  • Edit the entry by using the e key.
  • Add rd.break=<breakpoint> to the kernel command line (where <break-point> is one of the breakpoints described in man dracut.cmdline).
  • Press Ctrl+X to resume booting and press enter when prompted in order to start dracut shell.
  • Once in dracut shell you run whatever commands are needed.
  • Once finished, reboot with the reboot command. If you used chroot to chroot into the root filesystem, be sure to exit from the chroot before running the reboot command or it might not work.

In many cases, like for xfs_repair, rd.break=pre-mount will be enough.

Root Cause

OpenShift by default assigns an invalid password for core local user and therefore is not possible to log in by console until a correct password is assigned. Set or change the core user password feature via machineconfig is available since OCP 4.13. Refer to changing the core user password for node access for additional information.

SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.