Administrative Procedures for RHEL High Availability Clusters - Configuring z/VM SMAPI Fencing with fence_zvmip for RHEL IBM z Systems Cluster Members

Updated

Contents

Overview

Applicable Environments

  • Red Hat Enterprise Linux (RHEL) 7 or newer with the High Availability Add-On
  • Using IBM z Systems z/VM guests running RHEL 7 or newer as High Availability cluster members

Situations Where This Procedure May Be Useful

  • A RHEL High Availability cluster is being created from IBM z Systems z/VM guests, and STONITH configuration needs to be completed
  • It is preferred that the cluster manage these members via power fencing using the z/VM Systems Management Application Programming Interface (SMAPI), instead of (or in addition to) other methods like sbd or a storage-based STONITH agent.

What This Procedure Accomplishes

This procedure will result in the cluster having control over the power on/off state of cluster members, and once configured the cluster will use that control to reboot or power off a member if it becomes unresponsive or experiences some other cluster failure.

Procedure: Configuring z/VM SMAPI Fencing

Task: Setup a z/VM user to interact with the SMAPI

The SMAPI must accept certain requests from a fence user account for each z/VM LPAR hosting a cluster node. This fence user must have the required permissions for those requests, and the cluster's fence device will need to be configured with that user's credentials.

There can be one fence user with the required permissions for all images in the cluster managed by this SMAPI, or one fence user per image if that's preferred.

First, create a name list containing the required permissions in the name list file. The name list file is located in the source SFS directory (VMSYS:VSMWORK1); the default file name is VSMWORK1 NAMELIST. For this example, we'll create a name list called ZVM_FENCE.

:nick.ZVM_FENCE
:list.
IMAGE_ACTIVATE
IMAGE_DEACTIVATE
IMAGE_STATUS_QUERY
CHECK_AUTHENTICATION
IMAGE_NAME_QUERY_DM

To grant the permissions in the name list to the fence user, edit the SMAPI authorization file. The authorization file is located in the source SFS directory (VMSYS:VSMWORK1); the default file name is VSMWORK1 AUTHLIST. Set the user name in the first column; the image, image list, or "ALL" in the second column; and ZVM_FENCE (or whatever you called your name list) in the third column. For this example, we'll authorize a user called ZCLUSTER for all images managed by this SMAPI.

NOTE: This file is sensitive to starting location of the columns. Follow the exact format of existing entries there.

Column 1                                     Column 66                                    Column 131
|                                            |                                            | 
|                                            |                                            | 
V                                            V                                            V 
DO.NOT.REMOVE                                DO.NOT.REMOVE                                DO.NOT.REMOVE
ZCLUSTER                                     ALL                                          ZVM_FENCE

Refer to the official IBM documentation for complete details on Content from www.ibm.com is not included.name lists and Content from www.ibm.com is not included.SMAPI authorization.


Task: Collect information that will be needed by the cluster

For each z/VM guest running RHEL that will operate as a member of the cluster, have the following information ready:

NOTE: The <STRING> in parentheses represents how we will refer to this field in the upcoming task.

  • The IP address or hostname of the SMAPI for that z/VM LPAR (<NETWORK ADDR>)
  • The user that was configured to have the IMAGE_OPERATIONS function permission (<USER>)
  • The password for that user (<PASSWORD>
  • The name of the image, as known to z/VM (<IMAGE>)
  • The corresponding name that the cluster knows the member as, as reflected in pcs status or /etc/corosync/corosync.conf. (<NODENAME>)

NOTE: Confirm if your ZVM infrastructure supports HTTPS. If not, you may need to disable SSL (disable_ssl="1") in the stonith device definition.


Task: Create STONITH device(s) in the cluster

For every z/VM LPAR SMAPI that manages a guest running in this cluster, create a fence_zvmip STONITH device in the cluster's configuration, using the information gathered in the last step. The pcmk_host_map attribute on this STONITH device should capture the nodename-to-image mapping only of images managed by this SMAPI (separate by semi-colons if there are multiple).

# # NOTE: Only include the relevant nodes/images for this SMAPI in pcmk_host_map
# # RHEL 8 or later
# pcs stonith create <stonith device label> fence_zvmip ip=<NETWORK ADDR> username='<USER>' password='<PASSWORD>' pcmk_host_map="<NODENAME>:<IMAGE>[;<NODENAME>:<IMAGE>...]"

# # RHEL 7, or in the case of an error stating that there are unsupported options
# pcs stonith create <stonith device label> fence_zvmip ipaddr=<NETWORK ADDR> login='<USER>' passwd='<PASSWORD>' pcmk_host_map="<NODENAME>:<IMAGE>[;<NODENAME>:<IMAGE>...]"

For example, with the following conditions:

  • Two z/VM LPARs with SMAPIs at zvm-smapi1.example.com and zvm-smapi2.example.com,
  • Each SMAPI accepts requests of the relevant function from user "ZCLUSTER" and password "ZPASSWORD" for the guest image(s) assigned to this LPAR,
  • Each SMAPI manages two out of four nodes, and
  • Guest image names RHELHA-[1-4] and corresponding cluster-nodenames rhelha-node[1-4].example.com:
# # RHEL 8 or later
# pcs stonith create zvm-smapi1 fence_zvmip ip=zvm-smapi1.example.com username='ZCLUSTER' password='ZPASSWD' pcmk_host_map="rhelha-node1.example.com:RHELHA-1;rhelha-node2.example.com:RHELHA-2"
# pcs stonith create zvm-smapi2 fence_zvmip ip=zvm-smapi2.example.com username='ZCLUSTER' password='ZPASSWD' pcmk_host_map="rhelha-node3.example.com:RHELHA-3;rhelha-node4.example.com:RHELHA-4"

# # RHEL 7, or in the case of an error stating that there are unsupported options
# pcs stonith create zvm-smapi1 fence_zvmip ipaddr=zvm-smapi1.example.com login='ZCLUSTER' passwd='ZPASSWD' pcmk_host_map="rhelha-node1.example.com:RHELHA-1;rhelha-node2.example.com:RHELHA-2"
# pcs stonith create zvm-smapi2 fence_zvmip ipaddr=zvm-smapi2.example.com login='ZCLUSTER' passwd='ZPASSWD' pcmk_host_map="rhelha-node3.example.com:RHELHA-3;rhelha-node4.example.com:RHELHA-4"

Special consideration: Guests from multiple LPARs in an SSI cluster:

If using multiple LPARs in an SSI cluster, then in the above task there should be one fence_zvmip per LPAR.


Optional follow-up task: Create STONITH topology levels for redundant methods

If this cluster is configured to use redundant STONITH methods for any given node - such as sbd with poison-pill fencing as primary and fence_zvmip as backup - then these new fence_zvmip devices must be placed in a topology (STONITH levels) corresponding to that design.


Task: Adjust z/VM Dirmaint duration OR STONITH operation timeouts to avoid disruptions


SBR
Category
Components
Article Type