What are the default value of system-reserved node resources and eviction in OpenShift 4 ?

Solution Verified - Updated

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4

Issue

  • What is the default setting of "reserved" in OpenShift 4?
  • What is the default setting of "eviction" in OpenShift 4?

Resolution

Disclaimer: Links contained herein to external website(s) are provided for convenience only. Red Hat has not reviewed the links and is not responsible for the content or its availability. The inclusion of any link to an external website does not imply endorsement by Red Hat of the website or their entities, products or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) the external site or content.

Reserved resource setting includes --kube-reserved and --system-reserved in OpenShift. The default values in OpenShift 4 are:

Eviction resource limit setting include --eviction-hard and --eviction-soft in OpenShift. The default values in OpenShift 4 are:

  • For the eviction-hard: default setting values are included at Content from github.com is not included.defaults_linux.go of OpenShift release-4.16 . OpenShift will evict a running Pod when the left resource is less than setting value:

    var DefaultEvictionHard = map[string]string{
	"memory.available":  "100Mi",
	"nodefs.available":  "10%",
	"nodefs.inodesFree": "5%",
	"imagefs.available": "15%",
}
    • For the eviction-soft: None.

IMPORTANT NOTE: the default setting value of reserved and eviction could be different by OpenShift version and Machine Config Operator version.

Root Cause

The default reservation is expected to be sufficient for most configurations, but it could be not enough in some cases. For additional recommendations, refer to Which amount of CPU and memory are recommended to reserve for the system in OpenShift 4 nodes?

The system-reserved is only used for the processes running from /system.slice/, and in OpenShift 4 they are only node critical processes such as CRI-O, kubelet, etc. It's possible to see the entire list of processes executing the following command within the nodes:

# systemd-cgls /system.slice/

Diagnostic Steps

Check the /etc/kubernetes/kubelet.conf file in the nodes:

$ oc get nodes
[...]
$ oc debug node/[node-name]
[...]
# chroot /host bash
# cat /etc/kubernetes/kubelet.conf

Look through the output and find the systemReserved values:

[...]
systemReserved:                                                                                                                            
  cpu: 500m                                                                                                          
  memory: 1Gi
  ephemeral-storage: 1Gi
[...]

Check the processes included in the reservation:

$ oc debug node/[node_name]
[...]

sh-4.4# chroot /host bash
# systemd-cgls /system.slice/
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.