oc debug node Fails When a Default nodeSelector is Defined

Solution Verified - Updated

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4

Issue

  • oc debug node fails when default nodeSelector is defined (either globally and/or at project level) and you attempt to run it on a node which does not match the default nodeSelector.

Resolution

  1. Create a new namespace with an empty default nodeSelector:
$ oc adm new-project ${project_name} --node-selector=""

Or alternatively, annotate an existing project:

$ oc annotate project ${project_name} "openshift.io/node-selector"=""
  1. Then, whenever you run an oc debug node, run it in that namespace like this:
# oc debug -n ${project_name} node/<node>

Root Cause

Disclaimer: Links contained herein to external website(s) are provided for convenience only. Red Hat has not reviewed the links and is not responsible for the content or its availability. The inclusion of any link to an external website does not imply endorsement by Red Hat of the website or its entities, products, or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) the external site or content.

When oc debug node is run, it creates a pod in the current namespace. In the event that a default nodeSelector is defined, that default nodeSelector will be enforced on the newly created pod. As a result, the pod does not schedule.

This is by design starting from RHOCP OCPv4.6.28 as stated by KCS#6152592, as the creation of additional namespace for debugging was coming from solving another This content is not included.BZ#1812813. After that, it was decided to revert that change, and that happened in the Content from github.com is not included.Pull Request#668.

Diagnostic Steps

oc debug node may fail in a way similar to this:

$ oc debug node/ip-10-0-0-00ab-abc-0computeinternal
Starting pod/ip-10-0-0-00ab-abc-0computeinternal-debug ...
To use host binaries, run `chroot /host`

Removing debug pod ...
Error from server (BadRequest): container "container-00" in pod "ip-10-0-0-00ab-abc-0computeinternal-debug" is not available
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.