logical volumes are in partial state due to missing physical volumes after disk migration

Environment

• Red Hat Enterprise Linux
• lvm

Issue

• logical volumes are in unknown state due to missing physical volumes
• pvs output reported missing/unknown devices

Resolution

• Identify and restore the missing physical volume to the system.
• Verify and confirm whether the physical volumes that are expected to be in the system are all present.
• Review the metadata of the vg (/etc/archive and /etc/backup). Identify the UUID of the pv that has reported missing.
• Once the pv is restored, the logical volumes based on the pv will be seen in normal and activated state.

Root Cause

• A single volume group comprise of one or more physical volumes. During VG migration from one system to another , one physical volume was mistakenly not added to the destination node which resulted the corresponding lvs in partial/unknown state

Diagnostic Steps

• Check the pvs and lvs output and confirm if there is any warning stating there are physical volume missing.
• Example

pvs_-a_-v

  Couldn't find device with uuid 3deb3c95-xxxx-xxxx-xxxx-xxxx-cecF-FB3l5H.
    There are 1 physical volumes missing.
    There are 1 physical volumes missing.      

• pvs output will throw warning 'There are 1 physical volumes missing’ and the missing pv. In the above example the Missing PV UUID is 3deb3c95A-xxxx-xxxx-xxx-xxxx-xxxx-FB3l5H. This is part of the vg SXG_vg00.

  Couldn't find device with uuid 3deb3c95-xxxx-xxxx-xxxx-xxxx-xxxx-FB3l5H.
    There are 1 physical volumes missing.
    There are 1 physical volumes missing.
..
   PV                      VG                    Fmt  Attr PSize   PFree  DevSize PV UUID                                PMdaFree  PMdaSize  #PMda #PMdaUse 1st PE
  /dev/mapper/disk-sda1                              ---       0      0  200.00m                                               0         0      0        0      0
..
  unknown device          SXG_vg00 lvm2 a-m   39.51g     0       0  3deb3c95-xxxx-xxxx-xxxx-xxxxcecF-FB3l5H        0         0      0        0   1.00m

• The /etc/lvm/archive files shows the VG in question and the corresponding lvs were created on a different host and not the same one where the VG exist currently. An analogy would be currently the VG exist in hostx1 however the VG was originally created in hostx2.

pv0 {
                        id = "3deb3c95-xxxx-xxxx-xxxx-vtFG-cecF-FB3l5H"
                        device = "unknown device"       # Hint only    <<<<<<<

                        status = ["ALLOCATABLE"]
                        flags = ["MISSING"]                <<<<<<<<<<<<<<<<<
..
                }

      lvol00 {
                        id = "4acb4c89-xxxx-xxxx-xxxx-xxxx-xxxx-xxxe0e"
                        status = ["READ", "WRITE", "VISIBLE"]
                        flags = []
                        creation_host = "hostx2"              <======= actual host where the VG created
..
                                ]
                        }

• The creation host is hostx2 which is different . the host which was reporting missing PV has a different name.
• Verify if there was any migration done because of which physical volumes were mistakenly not added to the destination host

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.