Getting Journal Logs from the OpenShift 4.x Kubelet

Environment

• Red Hat OpenShift Container Platform (RHOCP)
  • 4

Issue

• I can't SSH to my node and need to know how to get access to the systems journal to investigate what is happening on my host.

Root Cause

OpenShift 4.x provides a shim in the kublets debug endpoints that allow you to pull journal logs from the system.

Diagnostic Steps

1. Using curl (primarily for bootstrap node)

   • Determine the IP or Hostname of your bootstrap node

   • You can get journal logs out of the kubelet directly by doing the following:

     $ curl --insecure --cert ${INSTALL_DIR}/tls/journal-gatewayd-client.crt --key ${INSTALL_DIR}/tls/journal-gatewayd-client.key 'https://${BOOTSTRAP_IP}:19531/entries?follow&_SYSTEMD_UNIT=bootkube.service'
     

2. Using oc adm node-logs (for after an install has completed)

   • Using oc adm node-logs you can access journald unit logs, or host file paths (under /var/log) by running one of the following commands:

   1. File Path Example

      • Note: only pulls logs or directory listing from files under host file /var/log on the denoted host or host group.

      $ oc adm node-logs --role=master --path=openshift-apiserver
      

   2. Journald Unit Example

      $ oc adm node-logs --role=master -u kubelet
      

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.