Pod placement not equally done across multiple zones

Environment

• Red Hat OpenShift Container Platform 3.x

Issue

• In some cases, pods gets scheduled in the same zone even if two identical zones with identical pods are available. We are using custom labels and have adjusted scheduler.json to implement anti-affinity based on these labels. The scheduler.json was extended with the following entry to achieve service-anti-affinity.

[...]
        {
            "argument": {
                "serviceAntiAffinity": {
                    "label": [ "datacenter" ]
                }
            },
            "name": "Datacenter",
            "weight": 10
        }
[...]

Resolution

Service-Anti-Affinity based on custom labels is not recommended to achieve equal distribution of pods across multiple zones. The Red Hat OpenShift Container Platform scheduler is at this point not able to fully achieve this due to various reason but in general due to the way it works by default. To achieve proper pod distribution across multiple zones it's highly recommended to use failure-domain.beta.kubernetes.io/region and failure-domain.beta.kubernetes.io/zone node label as those were specifically created for such a use-case. When installing Red Hat OpenShift Container Platform with a cloudprovider enabled, Red Hat OpenShift Container Platform will automatically set failure-domain.beta.kubernetes.io/region and failure-domain.beta.kubernetes.io/zone based on the information supplies in the inventory file. If the labels are not set because no cloudprovider was specified, it's recommended to apply the labels if pod distribution across multiple zones is required.

Red Hat OpenShift Container Platform will automatically spread the pods in a replication controller or service across nodes in a single-zone Red Hat OpenShift Container Platform Cluster. With multiple-zone Red Hat OpenShift Container Platform Clusters, this spreading behavior is extended across zones via SelectorSpreadPriority. This is a best-effort placement, and so if the zones in your cluster are heterogeneous pod placement might get impact which could cause unequal spreading.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.