How do I configure my highly available cluster to enable access to storage devices managed by fence agents in RHEL 6 or 7?

Solution Unverified - Updated

Environment

  • Red Hat Enterprise Linux (RHEL) 6 or 7 with the High Availability Add On
  • cman clusters with or without pacemaker
  • A fence agent in use by one or more cluster nodes which controls access to a resource, such as storage, that must be enabled when a node joins the cluster. Agents provided by Red Hat that require such functionality are:

Issue

  • Which fence agents should include an <unfence> section in their /etc/cluster/cluster.conf on RHEL 6?
  • Nodes are not being "unfenced" by fence_scsi when they join a pacemaker cluster
  • Do I need to configure something special so that the cluster enables access to the shared storage devices managed by my fence devices when a node joins?
  • What storage fencing options are available?

Resolution

cman clusters without pacemaker (RHEL 6 only)


Define a device in the `` section within the `` section in `/etc/cluster/cluster.conf` with the necessary parameters to enable access to the resource(s) managed by the fence devices. For example: 
<?xml version="1.0"?>
<cluster name="exampleCluster" config_version="1">
   <clusternodes>
         <clusternode name="node1.example.com" nodeid="1">
                  <fence>
                             <method name="1">
                                        <device name="scsi"/>
                             </method>
                  </fence>
                  <unfence>
                             <device name="scsi" action="on"/>
                  </unfence>
         </clusternode>
         <clusternode name="node2.example.com" nodeid="2">
                  <fence>
                             <method name="1">
                                        <device name="scsi"/>
                             </method>
                  </fence>
                  <unfence>
                             <device name="scsi" action="on"/>
                  </unfence>
         </clusternode>
    </clusternodes>
    <cman two_node="1" expected_votes="1"/>
    <fencedevices>
           <fencedevice name="scsi" agent="fence_scsi"/>
    </fencedevices>
</cluster>
Notes:
  • The unfence section does not support "methods" as the fence section does. All devices that must be enabled on startup should be defined here in a single list.
  • Nodes will unfence themselves automatically when the cman service starts
  • Nodes can be unfenced manually using fence_node -U <nodename>
  • Be sure to configure the action attribute in the unfence device, as by default most agents will execute either "reboot" or "off".

pacemaker clusters (RHEL 6 or 7)


In RHEL 6 with `pacemaker`, `/etc/cluster/cluster.conf` should [only contain the `fence_pcmk` definition that was created by `pcs cluster setup`](https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/Configuring_the_Red_Hat_High_Availability_Add-On_with_Pacemaker/index.html#s1-clustercreate-HAAR), and `unfencing` is configured in the cluster CIB via `pcs`. In RHEL 7, everything is configured in the CIB as well.

To configure a device to be "unfenced" automatically on startup (which will trigger the "on" action), configure the device with meta provides="unfencing". For example, at creation time:

# pcs stonith create scsi fence_scsi meta provides="unfencing"

Or after creation:

# pcs stonith update scsi meta provides="unfencing"

Root Cause

Certain fence agents used within High Availability clusters manage resources that must be in the "on" state while a node is a member of the cluster, and are set to the "off" state when it is removed from the cluster. The most common example of this is in storage-based fence devices, such as those that log in to a fibre-channel switch or that which manages SCSI reservations on shared storage devices. When a node joins, the agent must enable access to the devices in question, and when a node is removed from the cluster, its access is cut off from the devices.

This is handled through a concept called "unfencing", which is defined in /etc/cluster/cluster.conf for pure-cman clusters, and is configured as a meta option provides="unfencing" in the CIB of cman clusters with pacemaker, or pure-pacemaker clusters. Having such a definition results in the cluster turning the devices "on" when a node joins the cluster, so that an administrator does not need to do so manually, as was the case for certain agents in RHEL 5 High Availability clusters.

SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.