How to run iPerf network performance tests in OpenShift Container Platform 4 [RHOCP]

Environment

• Red Hat OpenShift Container Platform
  • 4

Issue

• Get network bandwidth between pods in the same node.
• Get network bandwidth between pods in different nodes.
• Get network bandwidth directly between two different nodes.

Resolution

Note: The quay.io/rjhowe/oc-k8s-netperf:latest image is not supported by Red Hat, and you are encouraged to build your own iperf image following KCS 5233541 - Testing Network Bandwidth in OpenShift using iPerf Container

Easier performance testing method can be found here: OpenShift 4 Network Performance Throughput Testing

From pod to pod on the same node

1. Create a project and create a service account with added privileges:

$ oc new-project iperf-test  
$ oc patch namespace iperf-test -p '{"metadata": {"annotations": {"openshift.io/node-selector": ""}}}'
$ oc create serviceaccount iperf
$ oc adm policy add-scc-to-user privileged system:serviceaccount:iperf-test:iperf

2. Pick a node and label it with iperf=test:

$ oc label node <node> iperf=test

3. Start the iperf server pod:

$ oc run iperf3-sdn-server  --image=quay.io/rjhowe/oc-k8s-netperf:latest --overrides='{"spec": {"tolerations": [{"operator": "Exists"}],"nodeSelector":{"iperf": "test"},"serviceAccountName": "iperf"}}' -- iperf3 -s

4. Take a note of the iperf server pod IP address, it will be needed when running the client pod:

$ oc get pods -o wide

5. Start the iperf client pod:

Test within the SDN:

$ oc run --rm -it iperf3-sdn-client  --image=quay.io/rjhowe/oc-k8s-netperf:latest --overrides='{"spec": {"tolerations": [{"operator": "Exists"}],"nodeSelector":{"iperf": "test"},"serviceAccountName": "iperf"}}'
~$ iperf3 -c <iperf3-sdn-server-pod-IP>

Test from the host network to the SDN:

$ oc run --rm -it iperf3-host-client  --image=quay.io/rjhowe/oc-k8s-netperf:latest --overrides='{"spec": {"hostNetwork": true,"tolerations": [{"operator": "Exists"}],"nodeSelector":{"iperf": "test"},"serviceAccountName": "iperf"}}'
~$ iperf3 -c <iperf3-sdn-server-pod-IP>

7. Clean up after finishing:

$ oc delete project iperf-test
$ oc label node <node> iperf-

From pod to pod on different nodes.

1. Create a project and create a service account with added privileges:

$ oc new-project iperf-test  
$ oc create serviceaccount iperf
$ oc adm policy add-scc-to-user privileged system:serviceaccount:iperf-test:iperf
$ oc patch namespace iperf-test -p '{"metadata": {"annotations": {"openshift.io/node-selector": ""}}}'

2. Choose a node and add the label iperf=server and a second node and add the label iperf=client:

$ oc label node <node1> iperf=server
$ oc label node <node2> iperf=client 

3. Start the iperf server pod:

$ oc run iperf3-sdn-server  --image=quay.io/rjhowe/oc-k8s-netperf:latest --overrides='{"spec": {"tolerations": [{"operator": "Exists"}],"nodeSelector":{"iperf": "server"},"serviceAccountName": "iperf"}}' -- iperf3 -s

4. Take a note of the iperf server pod IP address, it will be needed when running the client pod:

$ oc get pods -o wide

5. Start the iperf client pod:

$ oc run --rm -it iperf3-sdn-client  --image=quay.io/rjhowe/oc-k8s-netperf:latest --overrides='{"spec": {"tolerations": [{"operator": "Exists"}],"nodeSelector":{"iperf": "client"},"serviceAccountName": "iperf"}}'
~$ iperf3 -l 1240 -c <iperf3-sdn-server-pod-ip>
~$ iperf3 -u -l 1240 -b 100M -c <iperf3-sdn-server-pod-ip>

6. Clean up after finishing:

$ oc delete project iperf-test
$ oc label node <node1> iperf-
$ oc label node <node2> iperf-

Directly between two different nodes.

1. Create a project and create a service account with added privileges:

$ oc new-project iperf-test  
$ oc create serviceaccount iperf
$ oc adm policy add-scc-to-user privileged system:serviceaccount:iperf-test:iperf
$ oc create service nodeport iperf-host --tcp 5201:5201

2. Choose a node and add the label iperf=server and a second node and add the label iperf=client:

$ oc label node <node1> iperf=server
$ oc label node <node2> iperf=client 

3. Start the iperf server pod:

$ oc run iperf3-host-server --image=quay.io/rjhowe/oc-k8s-netperf:latest --overrides='{"spec": {"hostNetwork": true,"tolerations": [{"operator": "Exists"}],"nodeSelector":{"iperf": "server"},"serviceAccountName": "iperf"}}' -- iperf3 -s

4. Take a note of the iperf server pod IP address, it will be needed when running the client pod:

$ oc get pods -o wide

5. Start the iperf client pod:

$ oc run --rm -it iperf3-host-client --image=quay.io/rjhowe/oc-k8s-netperf:latest --overrides='{"spec": {"hostNetwork": true,"tolerations": [{"operator": "Exists"}],"nodeSelector":{"iperf": "client"},"serviceAccountName": "iperf"}}'
~$ iperf3 -c <iperf3-host-server-pod-ip> -p 5201

6. Clean up after finishing:

$ oc delete project iperf-test
$ oc label node <node1> iperf-
$ oc label node <node2> iperf-

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.