How to detach fibre channel storage from RHEV hosts after deleting it from RHEV Manager

Solution Verified - Updated

Environment

  • Red Hat Virtualization (RHV) 4
  • Fibre channel (FC) storage

Issue

  • How do I detach a fibre channel (FC) storage LUN I no longer need from my RHEV hosts after deleting it from the RHEV Manager?
  • I just removed a storage domain from the RHEV-Manager. How do I safely detach those multipath LUNs from my RHEV hosts?

Resolution

  • At the time of this writing, RHV-M does not orchestrate the removal of LUNs from deleted Storage Domains. The multipath devices and the underlying paths of the deleted Storage Domain LUNs will remain present on the host(s) until, either a reboot, or manual removal happens.

  • A Request for Feature Enhancement (RFE), Red Hat Bug This content is not included.1310330, was filed for this. The result was that, as of RHV 4.4.6, an Ansible playbook can be used to automate the cleanup process;

    # ansible-playbook  remove_stale_lun.yml

  • This role is already installed in the RHV-M manager and the user can run the role from the manager by creating the playbook Content from github.com is not included.remove_stale_lun.yml which calls this role. This example playbook is also available at /usr/share/ansible/collections/ansible_collections/redhat/rhv/roles/remove_stale_lun/examples/remove_stale_lun.yml in RHV manager.

  • The first play in the playbook will get the host inventory from the RHV-M API on the provided DC and will store that information in the runtime inventory group dc_hosts. The second play will remove the LUN(s) from the multipath and its underlying devices from the SCSI layer from all the discovered hosts.

  • Prior to this, either the hosts had to be rebooted or a manual procedure had to be performed on each host, most likely requiring a Support Case to be opened with Red Hat Technical Support to accomplish that.

  • An example of how to use this role can be found in the 4.4 documentation here

    # ansible-playbook --extra-vars "lun=<LUN>" /usr/share/ansible/collections/ansible_collections/ovirt/ovirt/roles/remove_stale_lun/examples/remove_stale_lun.yml

Historical data (RHEV 3 and prior to RHV 4.4.6):

  • Prior to RHEV 3.6, the Red Hat Virtualization storage daemon (vdsmd) could get delayed while trying to access these "failed" devices. This was addressed in internal Bug 880738. As a result, in RHEV 3.6 and later, vdsm will essentially ignore these devices.

  • Starting in RHEL 7 and vdsm-4.17.11-0.el7ev, it is safe to unzone the lun.
    Note: The remark in the first point still applies. Multipath structures will not be auto removed when unzoning the LUN.

  • The safest way to remove the LUN and update vdsm with this information before RHEL 7 and have vdsm-4.17.11-0.el7ev, is to do the following:

    • Delete the Storage Domain using RHEV-M.
    • Put the host in maintenance mode.
    • Unpresent the LUN from this host on the storage side.
    • Reboot this host to refresh the multipath definitions.
    • Repeat for the remaining hosts.

  • If this is a production environment and maintenance mode cannot be implemented right away, please contact Red Hat Support for assistance with a work-around procedure, which does not require putting the hosts in maintenance mode and can be done while vdsmd is running.

Root Cause

  • The process of LUN removal from the hosts is not directly performed by RHEV-M or vdsm, removal of these devices happens when the hosts are rebooted.
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.