Can a quorum disk be on a LVM volume in a RHEL High Availability cluster?

Solution Unverified - Updated

Environment

  • Red Hat Cluster Suite 4+
  • Red Hat Enterprise Linux 5 Advanced Platform (Clustering)
  • Red Hat Enterprise Linux Server 6 (with the High Availability Add on)

Issue

  • Can a quorum disk be on a LVM volume?
  • qdiskd fails to start when it can't find the correct device:
Jan 11 10:15:10 node1 qdiskd[3206]: <crit> Unable to match label 'qdisk' to any device

Resolution

Create/configure the quorum device to use a block device directly, as opposed to creating it on a shared LVM logical volume.

LVM logical volumes are not supported for usage as a quorum device.

Root Cause

The quorum disk is a disk device that is shared to all the cluster nodes to read and write to at the same time. The only supported way to share activated LVM volumes across multiple hosts at the same time is to use clvmd. One of the problems that can be encounted with the quorum device on clvmd volume is stated below.

In order to use a logical volume on multiple cluster nodes at once, it must be in a clustered volume group that is managed by clvmd. The service clvmd cannot start until the cluster has quorum, meaning any service that requires access to clustered logical volumes must wait until enough nodes are present to be quorate.

When QDisk is in use, there is a chicken-and-egg problem. Consider a standard two-node cluster with a quorum device, where everything has one vote apiece, and qdisks is configured to use a clustered logical volume. quorum would be 2 votes, so either both nodes or one node plus qdiskd must be online to gain quorum. However, the service qdiskd cannot start until the logical volume is made available by clvmd, and clvmd cannot start until quorum is gained. Since quorum cannot be gained without qdiskd started (unless the other node comes online), then the cluster node will not join the cluster.

For more information, see:

Diagnostic Steps

  • Find out what device the quorum disk is on, then see if that device or volume is managed by LVM.
# mkqdisk -L
mkqdisk v0.6.0
/dev/mapper/qvg-qlv:
/dev/qvg/qlv:
 Magic:                eb7a62c2
 Label:                qdisk
 Created:              Mon Jan 10 13:16:45 2011
 Host:                 server3.example.com
 Kernel Sector Size:   512
 Recorded Sector Size: 512

If the device is on an LVM logical volume and the cluster is not started yet, then this command may not identify any disk as the quorum device, so additional steps may be needed to start the daemons to make the logical volume available. If mkqdisk -L then shows a volume as the quorum device, this demonstrates the reason why its problematic to use LVM volumes: you must start all daemons before the volume can be activated for use by qdiskd, but you would require quorum before you can start the daemons, which may have required the quorum device in order to gain quorum.

Product(s)
Components
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.