Red Hat OpenShift Container Platform High Availability, and Recommended Practices

Updated

Overview

This document describes recommended practices for planning and deploying Red Hat OpenShift Container Platform clusters in a Highly Available and Fault Tolerant manner, spanning multiple sites (Datacenters or Regions of a Cloud Provider).

These additional articles describe common or recommended architecture for OpenShift Container Platform, many of the concepts captured by these documents are the foundation for the described recommendations. Please refer to them as needed:

Reference Articles

General Support Articles

CPU Architecture Support Information

Currently Red Hat OpenShift Container Platform is supported on x86_64, Power(ppc64le) and SystemZ (s390) CPU architectures.

Hardware Support Information

  • Red Hat OpenShift Container Platform 3.x is supported on any hardware that runs or certifies Red Hat Enterprise Linux 7 and meets the denoted CPU architecture requirements above.
  • Red Hat OpenShift Container Platform 4.x is supported on any hardware that runs or certifies Red Hat Enterprise Linux 7 or Red Hat Enterprise Linux 8 (to include Red Hat Enterprise Linux CoreOS) and meets the denoted CPU architecture requirements above.

Third Party Hardware/Software Integration

Due to the design of OpenShift there are a large number of integration possibilities, with networking (SDN solutions and routers), storage, and software deployments and applications. Where applicable our documentation will call out the support of these components or how patterns may support or integrate with OpenShift.

Deployment Recommended Practices

In any cluster deployment planning process, it is critical that you read and review, the documented 3.x planning guide, or 4.x Architecture as it outlines key questions and considerations you need to review and consider when deploying a cluster.

Red Hat Recommends that when installing OpenShift Container Platform 3.x or OpenShift Container Platform 4.x that it be deployed to a single site (datacenter) or cloud region. Red Hat does not recommend deploying an OpenShift Container Platform Cluster that spans multiple sites and that if you need to be in multiple data centers or regions that, you deploy multiple clusters (one per region/site) and use tools like This content is not included.Advanced Cluster Management (ACM) to mange these clusters and the deployments on them.

Infrastructure as a Service (IaaS) or Cloud Provider

While Red Hat does not promote or endorse any Cloud Provider, we do provide 2 enterprise-grade, IaaS platforms (Red Hat OpenStack Platform and Red Hat Enterprise Virtualization) that both provide the needed functional requirements for hosting VM's, and for highly available and a fault tolerant cluster we recommend selecting an IaaS or Cloud Provider that allows you access to multiple racks/availability zones, to give you the highest availability possible (at a single site/region).

Please refer to 3.x cloud provider considerations or 4.x platforms when ensuring you have the needed prerequisites/install notes meet for your IaaS or Cloud Provider.

Sizing the Cluster

When trying to decide how your OpenShift Container Platform cluster should be deployed, there are 2 areas where you need to focus, the Control Plane (Masters, Infra, and Etcd Nodes) and the Data/Compute Plane (Application Nodes).

Control Plane (Masters, Infra, and Etcd)

As denoted in Red Hat OpenShift Container Platform Support Policies for Deployments Spanning Multiple Sites(Data Centers/Regions), we don't confine you to a minimum or maximum number of Masters/etcd (combined) nodes (for 3.x) but we do recommend 3 nodes, as it provides a good balance between availability and scale, while still being easy to manage and maintain. With 4.x, 3 masters/etcd (combined) nodes is the default and minim number required to run a cluster.

  • Note: (3.x only) In addition to this (stated above), we don't recommend more than 5 sites or Master/Etcd nodes, as large clusters with more members of etcd have known performance issues.

When trying to plan the number of infra nodes you are going to need/use you need to consider what will run on the infra nodes, routers, logging, metrics, dynamic storage tools/providers, etc. Red Hat doesn't set limits on the number of nodes you can denote as 'infra' nodes, however the number of nodes often depends on what services you choose to run on these hosts, and the HA/Fault tolerance, and performance characteristics you want each service to have.

Generally speaking 3 to 9 nodes, with sufficient CPU, memory, and storage to run all or a subset of these tools is what is recommended.

For more information on properly sizing these nodes, see the following:

3.x:

4.x

Data/Compute Plane (application nodes)

When trying to plan or determine how many application nodes you will need or use, you can use the formulas defined by our This page is not included, but the link has been rewritten to point to the nearest parent document.sizing guide. These formulas are often important when it comes to determining your networking host subnet range, or ensuring that your nodes fit within the defined subnet for the cluster.

Network Recommendations

When possible we don't recommend the underlying host subnet use vxlan subnetting, as OpenShifts SDN also uses vxlan subnetting and will create a performance bottleneck with double encapsulation (~30 throughput loss). If your underlying host subnet uses vxlan subnetting (like with Red Hat OpenStack Platform), we recommend using an alternative OpenShift networking plugin.

Storage Recommendations

As a general rule, we suggest that storage provided to etcd, and/or hosts follow our defined storage recommendations, in our performance guide.

Storage for infrastructure components

Registry: Object storage solutions, is recommend if available. Its normally an optional component in almost all cloud providers, and or IaaS offerings. Due to its servicing mechanisms with the registry, it often makes the best recommendation as the selection for storage as it pertains to the registry.

Logging: Fast, low-latency storage solutions (block storage backed by fiber or high throughput connections) are what is recommended for use with this component. If this can be provided by the dynamic provisioner chosen for applications it is acceptable to use that. However often time an admin defined PV will be used for this component.

  • Note: We do not, recommend or support the use of NFS (due to file locking constraints), with this component.

Metrics: Fast, low-latency storage solutions (block storage backed by fiber or high throughput connections) are what is recommended for use with this component. If this can be provided by the dynamic provisioner chosen for applications it is acceptable to use that. However often time an admin defined PV will be used for this component.

  • Note: We do not, recommend the use of NFS (due to file locking constraints limiting throughput), with this component.

Dynamic Provisioning Recommendations for Applications

When possible its recommend that storage for applications, be provided by a dynamic storage provider. For this form of storage, it's critical, in a multi-site topology that such storage is accessible by all nodes, across all sites as applications can move freely between sites, to maintain application consistency.

Unsupported Items

The following features/scenarios will result in an unsupported cluster deployment.

Items marked for Technology Preview indicate that while the item is presently unsupported, Red Hat is working to fully support the feature in a future release of Red Hat Enterprise Linux.

Product(s)
Category
Article Type