How is the cluster-wide proxy setting applied to OCP nodes?

Environment

• Red Hat OpenShift Container Platform (RHOCP)
  • 4

Issue

• How do CRI-O/Kubelet utilize the cluster-wide proxy settings?
• When cluster-wide proxy settings are applied to the OCP cluster, how does CRI-O/Kubelet utilize these settings?
• How does cluster-wide proxy setting take into effect on OCP nodes?

Resolution

• Typically, it's the CRI-O container runtime that utilizes the proxy settings configured through the Openshift cluster-wide proxy. After applying the cluster wide proxy, the machine-config-daemon on OCP nodes creates a /etc/mco/proxy.env file with the required proxy settings. Then the machine-config-daemon creates another systemd drop-in file at /etc/systemd/system/crio.service.d/10-mco-default-env.conf for the CRI-O systemd service which loads the proxy configuration file /etc/mco/proxy.env as an environment variable.

Disclaimer: The IP addresses and hostnames provided here are for convenience only and are sourced exclusively from the Red Hat LAB environment.

Diagnostic Steps

Sample example from a test system

• Cri-o container runtime consumes the environment variable present in the /etc/systemd/system/crio.service.d/10-mco-default-env.conf file which is created by the machine-config-daemon.

# ls /etc/systemd/system/crio.service.d/
01-kubens.conf  10-mco-default-env.conf  10-mco-default-madv.conf  10-mco-profile-unix-socket.conf  20-nodenet.conf

# systemctl cat crio.service 

# /usr/lib/systemd/system/crio.service
[Unit]
Description=Container Runtime Interface for OCI (CRI-O)
Documentation=https://github.com/cri-o/cri-o
Wants=network-online.target
Before=kubelet.service
After=network-online.target

[Service]
Type=notify
EnvironmentFile=-/etc/sysconfig/crio
Environment=GOTRACEBACK=crash
ExecStart=/usr/bin/crio \
          $CRIO_CONFIG_OPTIONS \
          $CRIO_RUNTIME_OPTIONS \
          $CRIO_STORAGE_OPTIONS \
          $CRIO_NETWORK_OPTIONS \
          $CRIO_METRICS_OPTIONS
[...]

# /etc/systemd/system/crio.service.d/10-mco-default-env.conf
[Service]
EnvironmentFile=/etc/mco/proxy.env

• Content of /etc/mco/proxy.env file after the proxy settings was applied through the MCD.

[root@worker-0 ~]# cat /etc/mco/proxy.env 
# Proxy environment variables will be populated in this file. Properly
# url encoded passwords with special characters will use '%<HEX><HEX>'.
# Systemd requires that any % used in a password be represented as
# %% in a unit file since % is a prefix for macros; this restriction does not
# apply for environment files. Templates that need the proxy set should use
# 'EnvironmentFile=/etc/mco/proxy.env'.
HTTP_PROXY=http://192.168.x.x:3128
HTTPS_PROXY=http://192.168.x.x:3128
NO_PROXY=.cluster.local,.svc,10.128.0.0/14,127.0.0.1,172.30.0.0/16,192.168.100.0/24,api-int.ocp13.ocplabs.bm,localhost,ocplabs.bm

• The Kubelet itself does not directly use the proxy setting. Instead, it is the CRI-O container runtime that accesses the proxy setting from the /etc/mco/proxy.env file, which is generated by the machine-config-daemon.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.