"The specified vg_access_mode doesn't match the lock_type on VG metadata!" observed after creating `LVM-activate` cluster resources using `vg_access_mode=system_id`

Environment

• Red Hat Enterprise Linux 8, 9
• High Availability Pacemaker Cluster

Issue

• After creating the cluster resource LVM-activate using the vg_access_mode=system_id option, it gives an error pointing the vg_access_mode as incorrect and the LVM-activate resource fails to start:

  # pcs status
  ...
  Full List of Resources:
    * Resource Group: splunkdb:
      * lvm-rsc   (ocf::heartbeat:LVM-activate):   Stopped
      * fs_rsc    (ocf::heartbeat:Filesystem):     Stopped
   
  Failed Resource Actions:
    * lvm-rsc_start_0 on node1 'not configured' (6): call=32, status='complete', exitreason='The specified vg_access_mode doesn't match the lock_type on VG metadata!', last-rc-change='Fri May  3 10:38:41 2024', queued=0ms, exec=201ms
  

Resolution

• Note: If this issue is observed on an LVM-Activate resource with the options activation_mode=shared and vg_access_mode=lvmlockd then, refer to the following article:

  • "The specified vg_access_mode doesn't match the lock_type on VG metadata!" observed after creating LVM-activate cluster resources using vg_access_mode=lvmlockd

This issue usually happens when the VG was created ahead of starting the configuration of active/passive Pacemaker cluster and system_id_source was at its default value i.e. set to none.

1. Be sure that each node includes the following in /etc/lvm/lvm.conf:

   system_id_source="uname"
   

2. Manually add the system ID of the VG under cluster contrl using the current hostname (to be executed from any of the cluster node):

   # vgchange --systemid $(uname -n) <vg name>
   

3. System ID can be verified using the following vgs command from the same node where step#2 was executed:

   # vgs -o+systemid
   

4. After the system ID is manually set, the cluster will manage assign the appropriate system ID as the resource fails over from that point forward. Run a resource cleanup so the cluster forgets the resource failed operations and tries to start it again.

   # pcs resource cleanup <resource-name>
   

Root Cause

The issue arises because the system_id_source parameter is set to uname along with LVM-activate resource is configured with vg_access_mode=system_id. However the VG which is managed by that LVM-activate resource is missing the lock_type in VG metadata. It usually happens when the VG to be added under cluster control is created before making the necessary updates in /etc/lvm/lvm.conf file & creation of LVM-activate resource to manage the activation of VG.

Diagnostic Steps

The cluster reports for the following failure:

Failed Resource Actions:
  * lvm-rsc_start_0 on node1 'not configured' (6): call=32, status='complete', exitreason='The specified vg_access_mode doesn't match the lock_type on VG metadata!', last-rc-change='Fri May  3 10:38:41 2024', queued=0ms, exec=201ms

It can be found in the logs that the start operation of the LVM-activate resource resulted in a failure with below events:

May  3 10:25:48 node1 pacemaker-controld[2129]: notice: Requesting local execution of start operation for lvm-rsc on node1
...
May  3 10:25:48 node1 LVM-activate(lvm-rsc)[1213740]: ERROR: The specified vg_access_mode doesn't match the lock_type on VG metadata!
May  3 10:25:48 node1 pacemaker-controld[2129]: notice: Result of start operation for lvm-rsc on node1: not configured (The specified vg_access_mode doesn't match the lock_type on VG metadata!)

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.