OpenShift Application Introspection and Debugging with 4.x

Updated

With many parts of OpenShift running on the cluster (as part of Kubernetes), how you debug or investigate an application or set of objects on the cluster becomes of crucial importance. This guide aims to generically define the basics of application introspection and debugging so that you know how to use the This page is not included, but the link has been rewritten to point to the nearest parent document.oc tool (or kubectl - as they should be interchangeable) chain to review what is happening in your cluster.

Cheat Sheet

Describing or Defining an Object

  • oc api-resources enumerates the resource types available in your cluster. This means you can combine it with oc get to actually list every instance of every resource type in a namespace:

    $ oc api-resources --verbs=list --namespaced -o name

    • This can then be paired with other commands/verbs, like get

      $ oc api-resources --verbs=list --namespaced -o name  | xargs -n 1 oc get --show-kind --ignore-not-found -n <namespace>

  • Once you know what resources exist in your cluster/namespace you can inspect these resources 

    $ oc get <object>  -n <namespace> -o [json|yaml]
$ oc describe <object>

To see how this can be used in action please refer to the following KCS Examples (common uses for debugging):

Getting Additional Critical Data Points

Throughout this section, we will be looking at specific objects (like pods), and defining what the critical data points are you want to collect when looking at an application having issues. All of these commands are namespaces scoped (meant to be run in the namespace you are debugging or with -n <namespace> appended to the command.

  • OpenShift uses the resources in the config.openshift.io group to drive the individual operators needed to realize that configuration. To dump all those resources, you need to find them and get them. To do this you can run the following: 

    $ oc api-resources --api-group=config.openshift.io
$ oc get <resource> -o yaml
    • For more information to see how to read and understand what is provided by these resources please see KB 4007771

  • Generally, above we describe how to list resources and print information on them. One critical object that defines the health of the OpenShift components are the Operators that manage those components. Should you need to get one of the status: Available, Failing or Processing you can use the following to get this for all the operators in a namespace. 

    $ oc get clusteroperator -o yaml

  • To be more specific about the critical object that needs to be reviewed, over what the generic guidance (above) provides, support will be looking for information from the following objects: deployments, daemonsets, pods, configmaps, services 

    $ oc get <object> -o yaml

  • One primary thing you may want/need to do when debugging an application is get its logs. These are always pulled from a pod so the command syntax to do this will revolve around the pods that you can list. 

    $ oc  logs <pod>

  • In addition to logs, another place to look for information about what is happening to an application are the events

    • While these are not directly tied to a pod; they often will be tied to a pod or some other object tied to a pod. 
    $ oc get events -o yaml

  • Due to the way object references work it may be important to list out and have an understanding of what resources exist in a namespace.

    • Note: for secrets this only gives a summary (does not use -o yaml). It does not reveal any confidential information.
    $ oc get secrets
$ oc get pvcs -o yaml 
$ oc get pvs -o yaml

  • You can also use component endpoints to get critical data on these components.

SBR
Product(s)
Category
Article Type