Implementing Broadcast Mode in Red Hat Enterprise Linux Clustering and High Availability Environments

Updated

Issue

Using broadcast is fully supported on Red Hat Enterprise Linux(RHEL) 4 and RHEL 5.4+, but is only a Technology Preview on RHEL 6.

Red Hat Cluster Suite 4.x used broadcast by default, although it can be configured to use multicast rather than broadcast. In RHEL 5 and RHEL 6 the default is multicast.

Environment

  • Red Hat Enterprise Linux Server 5 (with the High Availability and Resilient Storage Add Ons)
  • Red Hat Enterprise Linux Server 6 (with the High Availability and Resilient Storage Add Ons)

Resolution

Support policies

Support polices - transport protocols

Configuration

Add the following attribute to the cman stanza of your /etc/cluster/cluster.conf.

<cman broadcast="yes"/>

Then propagate the changes to the cluster.conf to all the cluster nodes. The service cman will need to be restarted on all the cluster nodes. This means that a complete restart of the cluster service stack on all cluster nodes is required for the change to take affect.

To verify that the cluster is in broadcast mode then get the status of cman as show below. When the multicast address is 255.255.255.255 the it is in broadcast mode.

# cman_tool status | grep Multicast
Multicast addresses: 255.255.255.255

It is best practice to restrict the logical network so that it contains only cluster nodes so that only the cluster nodes will see the broadcast traffic that is generated. An ideal configuration will have the broadcast cast traffic on a private(heartbeat, fencing, locking) interface and all other traffic going over a non-private interface.

Broadcast mode is not suitable for deployments that require high performance locking. Such as GFS, GFS2, cmirror, any use of openais or corosync outside of RHEL that would require high performance message transport, such as Apache Qpid messaging.

The reason that broadcast mode is not suitable for GFS or GFS2 is because of the amount of network traffic that can occur. When broadcast mode is used the packets are sent to the network's broadcast address and received by all nodes on the network. The nodes must receive and ignore packets that aren't intended for them. This creates a great deal of unnecessary traffic on the network, and adds to the processing load of the nodes. Whereas multicast mode uses logic on the switch to send packets directly to any or all nodes in the cluster.

GFS and GFS2 also puts additional load on the network in the form of DLM messages that must go between the cluster nodes to manage filesystem locks. This traffic, combined with the other traffic necessary for cluster communication can overload the network or the cluster nodes.

  • If the nodes aren't able to properly process the token, then token loss will occur and nodes will be fenced.
  • If nodes aren't able to process DLM information properly the GFS and GFS2 filesystem may see lock contention, keeping data from being read from or written to disk.

For more information please see the following articles:

Product(s)
Category
Components
Tags