Cri-o is using a large amount of memory, how can I collect a performance profile ?

Environment

• Red Hat OpenShift Container Platform 4.x
• Cri-o

Issue

• How to troubleshoot Cri-o
• Cri-o memory usage is high
• How to profile Cri-o

Resolution

• Verify the memory usage for the Cri-o process by sshing to the node or using oc debug <node name> and running:

# ps -p $(pidof crio) -o pid,rss,vsz,cmd
    PID   RSS    VSZ CMD
   1665 112884 3120448 /usr/bin/crio

• To collect a performance profile from Cri-o using the Cri-o pprof endpoint, follow these steps:

# oc debug nodes/<node name>

# curl --unix-socket /host/var/run/crio/crio.sock http://localhost/debug/pprof/profile --output /host/var/tmp/crio_profile.pprof

# exit

then copy the profile file outside the node and share it to Red Hat Technical Support.

oc debug nodes/<node name> -- cat /host/var/tmp/crio_profile.pprof > crio_profile.pprof

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.