Should I configure my RHEL High Availability cluster nodes to access their fence devices over the same network interface as the cluster uses for communication or a separate network interface?

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux (RHEL) 5, 6, or 7 with the High Availability Add On
  • All fence devices in the cluster require a network connection in order to succeed

Issue

  • Which interface is best to use for cluster fence devices? The private cluster interconnect, or the public network?
  • Our organization has fence devices on a separate restricted network, and we can't move those connections over to our private cluster network. Will this work with the cluster software?
  • Are there advantages and disadvantages to splitting fence devices off to a separate network interface from the cluster?

Resolution

There are benefits and drawbacks to using either the cluster interconnect network interface or a separate network interface to access fence devices for a High Availability cluster. In general, a separate interface often offers a higher level of redundancy and reduces the possible points of failure for the entire cluster, but this may require a slightly more complex configuration to address the additional considerations it brings. In the end, the answer often depends on the availability of separate networks, where the fence devices are configured by default or by preference, and what failure scenarios are likely enough and critical enough to avoid in the configuration.

Fence Devices Accessed via Cluster Interconnect Interface

Pros:

  • Usually no need to configure the cluster to avoid fence races, considering an outage of the network that the nodes communicate over would also prevent them from accessing the fence devices. In other words, if the nodes lose contact with each other, there is no ability for both sides to fence the other simultaneously, and so no further measures are needed to prevent such an occurrence.

  • Simplicity of configuration of the host, in that multiple interfaces are not needed.

  • Lower cost, as this configuration would minimize the required network hardware

Cons:

  • If the single network used for cluster communications and fencing goes down, then all cluster operations will be blocked, leaving the cluster ineffective at managing high availability of resources. With no nodes in the cluster being able to communicate, and no nodes being able to reach the other fence devices, activity would halt.

  • If the fence devices are already configured to use a specific network, then configuring the cluster nodes to communicate on that same network may have drawbacks, such as:

  • increasing the load on that network, potentially disrupting other hosts
  • being susceptible to load on that network from other hosts disrupting or affecting the performance of cluster communications

Fence Devices Accessed via a Separate Interface

Pros:

  • Improved redundancy in the cluster, in that if the cluster interconnect network fails on one node or entirely, then access to the fence devices should not be entirely cut off. With an adequate configuration, this should leave at least one node in a position to maintain quorum, fence off remaining nodes, and continue to provide service in the cluster.

  • This arrangement aligns with how most environments are already configured by default, in that the fence devices would be accessible via a dedicated management network or via the "public" datacenter network, and the cluster communication would occur over a dedicated vlan or network. In other words, most organizations' standard network deployment could be left as-is for any deployed clusters.

Cons:

  • Requires configuring the cluster to avoid fence races and fence loops, which may conflict with other requirements that exist for the cluster, such as the need to have nodes automatically (re)join the cluster on boot.

  • Increased complexity of network configuration on the cluster nodes, as multiple interfaces are needed.

  • Requires additional network interfaces and additional network hardware and infrastructure.

SBR
Product(s)
Category

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.