Can I utilize clustered LVM volumes with clvmd and Pacemaker in a RHEL 6 or 7 High Availability cluster?

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux (RHEL) 6 or 7 with the Resilient Storage Add On
  • Pacemaker
  • Shared LVM volumes
  • lvm2-cluster

Issue

  • Does the Pacemaker LVM resource agent support clvmd?
  • Can I use clvmd in combination with Pacemaker and HA-LVM?
  • Can clvmd be used with Pacemaker for active/active access to volumes (i.e. when not using HA-LVM)?

Resolution

HA-LVM with RHEL 6 Pacemaker Cluster & clvmd (Active/Passive)

In RHEL 6 HA Cluster with Pacemaker , clvmd is not supported in combination with the LVM resource agent used by Pacemaker when using the volumes in an exclusive, active/passive manner. In other words, HA-LVM (active/passive LVM) configurations must use the tagging variant when Pacemaker is in use in RHEL 6.

If specifying exclusive=true on the resource, then locking_type should be set to 1 in /etc/lvm/lvm.conf:

# pcs resource create my_vg LVM volgrpname=my_vg exclusive=true

NOTE: While the recommendation when using a single-host volume in a cluster is to use HA-Tagging and exclusive=true set to your logical volume resource, in situations where you're using both single-host volumes (those hosting local filesystems such as ext4) and multi-host volumes (those volumes hosting a GFS2 distributed filesystem) you will need to manage both using CLVMD without the exclusive=true setting on your LVM resource. More information about this issue and the proper configuration can be found here

HA-LVM with RHEL 7 Pacemaker Cluster & clvmd (Active/Passive)


Using `clvmd` with the `LVM` resource agent used by pacemaker is functional and supported in RHEL 7 Pacemaker clusters, however the Red Hat recommendation is that if you are not using GFS2, you strongly consider instead using HA-LVM with Tagging. This recommendation is to prevent complications from using clvmd with single-host volumes such as:
  1. Increased overhead associated with maintaining volume group metadata between cluster nodes
  2. Required use of cmirror when using lvm mirroring in your environment, which introduces tremendous overhead
  3. Extra expense associated the Resilient Storage entitlement which is not required for an active/passive configuration
  4. Introduction of safety concerns resulting from unintentional activation and mounting of single-host volumes.

NOTE: Complete instructions for configuring Active/Passive LVM in pacemaker can be found here

Active/Active LVM

clvmd is fully supported in conjunction with Pacemaker in active/active configurations. In other words, if volumes need to be active concurrently on multiple nodes, then set locking_type=3 in /etc/lvm/lvm.conf and activate the "clustered" flag on the volume group with vgchange -cy <volume group>. If desired, active/active volume groups can be managed by the cloned LVM resource with exclusive=false:

# pcs resource create my_vg LVM volgrpname=my_vg exclusive=false clone

Managing clvmd and cmirrord daemons

This knowledge solution details how to manage availability of shared LVM volume groups utilized within the cluster. Instructions for managing clvmd and cmirrord daemons themselves can be found here.

SBR
Product(s)
Components
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.