Configuring a RHEL HA Cluster Fence Agent, 'fence_ibm_powervs', for an IBM Power Systems Virtual Server

Environment

• IBM Power Systems Virtual Servers
• Red Hat Enterprise Linux 8, 9 or 10 ( with the High Availability Add-on)
• Pacemaker

Prerequisites

• A supported version of Red Hat Enterprise Linux has been installed on each node
• RHEL HA has been installed, configured, and started without a configured fencing agent
• Meets Support Policies for RHEL High Availability Clusters - IBM Power Systems Virtual Server (PowerVS) Virtual Machines as Cluster Members

Configuring a RHEL HA cluster fence agent for an IBM Power Systems Virtual Server

1. Install the IBM cloud CLI, as documented in Content from cloud.ibm.com is not included.Installing the stand-alone IBM Cloud CLI.

   # curl -fsSL https://clis.cloud.ibm.com/install/linux | sh
   

2. Install the the IBM Power VS fence agent package.

   [root@clusternode ~]# yum -y install fence-agents-ibm-powervs
   

3. To perform a quick test, create user credentials on a local machine.

   a. Get one-time passcode to log in to CLI and API

   [user@workstation ~]$ ibmcloud login 
   

   b Get token.

    [user@workstation ~]$ ibmcloud iam oauth-tokens
    IAM token:  Bearer    
   XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX-XXXXXXXX-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXXXXXXXXXXXXXXXXXXXXX-XXXXX_XXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXXXXXXXXXXXXX-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXXX-XXXX_XX-XXX-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX-XXXXXXXXXXXXXXXXXXXXXX
   

4. Get the CRN and GUID from the IBM Cloud console.

   crn:v1:bluemix:public:power-iaas:us-east:a/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX::XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
   

5. List the available nodes using all the values from above.

   [root@clusternode ~]# fence_ibm_powervs --region [REGION] --token [IAM_TOKEN] --crn [CRN] --instance [GUID] -o list
   

   For example:

   [root@clusternode ~]# fence_ibm_powervs --region us-east --token "<SNIP>" --crn "crn:v1:bluemix:public:power-iaas:us-east:a/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX::" --instance XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX -o list
   aecbe71d-564c-4047-ba9c-13040eee8de6,bperkins-ibmpvs-rhel-8-4-0-20210503-0
   235667bd-156e-4b08-9c5f-524145125c02,bperkins-ibmpvs-rhel-9-0-0-20220401-0
   e34d62ad-6d96-4aa0-8ac3-2e5b1eb8043d,bperkins-ibmpvs-rhel-8-5-0-20211013-0
   6643bd9a-7ab9-4e8a-a568-217e99e4e26d,bperkins-ibmpvs-rhel-8-6-0-20220322-2
   12cb62c9-1681-40fa-a136-c70812df8116,bperkins-ibmpvs-rhel-8-6-0-20220322-1
   0fff1d58-d32a-4972-ae61-37fb1998af96,bperkins-ibmpvs-rhel-8-6-0-20220322-0
   4607eac9-7cf4-4840-9f59-dbe8c863acdf,bperkins-webui-2
   c9b566ad-cf01-4cf9-9604-33fd5891ef4a,bperkins-webui-1
   

6. Get the status (or other function) on an instance:

   [root@clusternode ~]# fence_ibm_powervs --region us-east --token "<SNIP>" --crn "crn:v1:bluemix:public:power-iaas:us-east:a/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX::" --instance XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX -o (on|off|reboot|status) -n XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
   

7. Create a STONITH agent.

   [root@clusternode ~]# pcs stonith create my_fence_ibm_powervs fence_ibm_powervs region=us-east token=<SNIP> crn=crn:v1:bluemix:public:power-iaas:us-east:a/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX:: instance=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX 'pcmk_host_map=n1:p1;n2:p2,p3'