How do I setup multipath on a system that already has LVM configured?

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux 4, 5, 6, 7, 8, 9
  • DM-Multipath

Issue

  • Is it possible to setup device mapper multipath easily on a system that already has LVM installed?
  • Does configuring DM multipath after LVM has been configured, require copying data?
  • Does the sequence of installing and configuring LVM and multipath matter?
  • How to configure existing lvm volume group to use DM-Multipath ?

Resolution

  • Configuring device mapper multipath after LVM has already been configured does not require copying data.

  • The sequence of configuring multipath and LVM does not matter.

  • This guide covers how to add auxiliary multipath storage to a system, not convert single path devices to multipath, or the conversion of a mounted device such as the root volume or swap.

  • Configuring multipath after LVM requires a few additional steps to ensure proper operation of LVM on multipath devices:

    1. Deactivate any logical volumes using underlying path devices of any multipath device (See verification steps to check for this type of logical volume).  These logical volume maps must be deactivated by using lvchange -an or vgchange -an. Once multipath is configured, the logical volumes will be re-activated using the multipath devices.

    2. Configure multipath using the procedure from our documentation:
      RHEL 6: DM Multipath Configuration and Administration Guide
      RHEL 7: Configuring and managing Device Mapper Multipath
      RHEL 8: Configuring device mapper multipath
      RHEL 9: Configuring device mapper multipath

    3. Configure LVM to work on top of multipath devices.  Change the following two configuration parameters in /etc/lvm.conf:

      1. filter
      2. preferred_names      
      • Changing filter line.      
        • You should change the filters line to "accept" multipath devices, and any other devices that contain LVM PVs, and reject all other devices. In particular, if the root volume and swap is on logical volumes backed by local disk devices, be sure to include these devices in the filter line. Existing LVM devices may be found by running the pvs command.

        • For example, the following filter line causes LVM to scan multipath devices with user_friendly_names, as well as local ATA devices, but skip scanning all other devices:

          filter = [ "a|/dev/mapper/mpath.*|", "a|/dev/hd|", "r|.*|" ]
          
        • For other filter line examples, see the /etc/lvm.conf file, or the This content is not included.Logical Volume Manager Administration Guide in the official Red Hat documentation

        • Changing the "preferred_names".

          • The preferred_names line only affects the output of LVM commands, but should be changed as follows, so that multipath devices are used before path devices.

          • In later versions on LVM2 you are no longer required to set this parameter, as multipath names will be used by default. However, as it causes no harm and is relevant to some older system, it remains here for completeness

          • This example causes multipath devices with user_friendly_names to be used:

            preferred_names = [ "^/dev/mpath/", "^/dev/mapper/mpath", "^/dev/[hs]d" ]
            
    4. Use pvscan to scan for LVM devices using the new filter settings. The output of 'pvscan' should show multipath devices.

    5. Use vgchange -ay or lvchange -ay to activate the logical volumes based on the multipath devices.

    6. Verify the LVM commands, pvs, vgs, and lvs, show all physical volumes, volume groups, and logical volumes correctly.  If the root ("/") volume is on a logical volume, make sure this displays properly.

      • As noted earlier, altering the root volume from single path to multipath is out of scope for this document. However, we should be sure that no unexpected changes have occurred on the root volume before continuing.
    7. For Rhel 7.7 and higher, edit /etc/dracut.conf and use

      add_dracutmodules+=" lvm multipath "
      
    8. Rebuild the initrd using This content is not included.How do I rebuild the initial ramdisk image in Red Hat Enterprise Linux?.

      • Again, altering the root multipath configuration is not the aim of this guide, we rebuild the initramfs here to ensure that the lvm.conf in the image and in /etc are in sync and to avoid surprises later

Verification Steps

  1. Verify the logical volume maps are created by using multipath devices and not underlying paths. One way to verify this is with dmsetup ls --tree.

    • Example of correctly setup logical volume map, containing multipath device:

      # dmsetup ls --tree
      vgtest-lvtest (253:5)
      `- mpath3 (253:2)
         `- (8:16)
         `- (8:0)
      
    • Example of incorrectly setup logical volume map, only containing path devices.  The following example shows a logical volume, "lvtest", mapped to an underlying path device, "sdb", of a multipath map, "mpath0"

      # multipath -ll mpath0
      mpath0 (360a980006e424b626d3463714b747750) dm-6 NETAPP,LUN
      [size=1.0G][features=0][hwhandler=0][rw]
      \_ round-robin 0 [prio=2][active]
       \_ 2:0:0:0 sda 8:0   [active][ready]
       \_ 3:0:0:0 sdb 8:16  [active][ready]
      # lvs -oname,devices vgtest/lvtest
        Found duplicate PV 32OmZC9Loea3Oo9I7utmlecwrItSNhPm: using /dev/sdb not /dev/sda
        LV     Devices    
        lvtest /dev/sdb(0)
      # dmsetup ls --tree | grep -A 4 "253:6"
      mpath0 (253:6)
       ├─ (8:16)
       └─ (8:0)
      vgtest-lvtest (253:2)
       └─ (8:16)
      
  2. Verify the LVM commands display the multipath devices.

    • Example:

      # pvs
      PV                VG         Fmt  Attr PSize  PFree
      /dev/hda2         VolGroup00 lvm2 a-    7.88G     0
      /dev/mpath/mpath3 vgtest     lvm2 a-    1.00T  1.00T
      /dev/mpath/mpath4 vgtest     lvm2 a-    1.00T  1.00T
      /dev/mpath/mpath5 vgtest     lvm2 a-    4.00T  4.00T
      

Root Cause

SBR
Category

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.