The srHook attribute is set to SFAIL in a Pacemaker cluster managing SAP HANA system replication, even though replication is in a healthy state

Environment

• Red Hat Enterprise Linux 8 for SAP Solutions

Issue

• SAP HANA scale-up: After a takeover, the hana_<sid>_sync_state attribute is set to SOK on the secondary node, but the hana_<sid>_site_srHook attribute is set to SFAIL.
• SAP HANA scale-out: After a takeover, the hana_<sid>_sync_state attribute is set to SOK on the secondary node, but the hana_<sid>_glob_srHook attribute is set to SFAIL.

Resolution

Fix the rules and configuration parameters in /etc/sudoers and /etc/sudoers.d.

Warning: Be sure to use the visudo command (or visudo -f <filename>) to edit /etc/sudoers or any file in the /etc/sudoers.d directory. The visudo command edits sudoers configuration files in a safe manner. See the visudo(8) man page for more details.

• The sudoers rules should be as described in the following documentation:
  • Scale-up: Automating SAP HANA Scale-Up System Replication using the RHEL HA Add-On
  • Scale-out: This content is not included.Red Hat Enterprise Linux HA Solution for SAP HANA Scale-Out and System Replication.
• The sudoers Defaults option requiretty must be one of the following:
  • unset (it's disabled by default in RHEL 8)
  • disabled globally (Defaults !requiretty)
  • disabled for the <sid>adm user (Defaults:<sid>adm !requiretty)
  • disabled for the appropriate commands
    • Scale-up: disabled for the <site>_SOK and <site>_SFAIL commands as defined in Automating SAP HANA Scale-Up System Replication using the RHEL HA Add-On for each site (Defaults!<site1>_SOK,<site1>_SFAIL[,...] !requiretty after the command aliases are defined)
    • Scale-out: disabled for the SOK and SFAIL commands as defined in This content is not included.Red Hat Enterprise Linux HA Solution for SAP HANA Scale-Out and System Replication (Defaults!SOK,SFAIL !requiretty after the command aliases are defined)

Root Cause

The HANA srConnectionChanged() hook failed to execute the command sudo /usr/sbin/crm_attribute -n hana_<sid>_glob_srHook -v <system_replication_status> -t crm_config -s SAPHanaSR in a HANA scale-out cluster (or the same command except with hana_<sid>_site_srHook in a scale-up cluster). The failure occurred due to sudo restrictions. Specifically, the option Defaults requiretty was set in /etc/sudoers.

According to the sudoers(5) man page:

     requiretty        If set, sudo will only run when the user is logged in to a real tty.  When this flag is set, sudo can only be run from a login session and not via other means such as cron(8) or cgi-
                       bin scripts.  This flag is off by default.

Since the HANA srConnectionChanged() hook runs the sudo /usr/sbin/crm_attribute command via a script (without a real tty), the command failed.

The solution, as described in the Resolution section, is to disable the requiretty option either globally, for the <sid>adm user, or for the specific commands that the hook executes.

Diagnostic Steps

1. The hana_<sid>_glob_srHook attribute is set to SFAIL, while the hana_<sid>_sync_state attribute is set to SOK and system replication is verified to be in a healthy state.

2. The /etc/sudoers file (or a file in /etc/sudoers.d) configures Defaults requiretty.

3. The HANA trace logs show that the hook is attempting to set the value of hana_<sid>_glob_srHook, but that it is failing (rc=256 when rc=0 is expected):

    [147045]{-1}[13/-1] 2021-09-01 11:16:20.293612 i ha_dr_SAPHanaSR  SAPHanaSR.py(00112) : SAPHanaSR CALLING CRM: <sudo /usr/sbin/crm_attribute -n hana_rh2_glob_srHook -v SFAIL -t crm_config -s SAPHanaSR> rc=256
    [147049]{-1}[11/-1] 2021-09-01 11:17:08.177424 i ha_dr_SAPHanaSR  SAPHanaSR.py(00112) : SAPHanaSR CALLING CRM: <sudo /usr/sbin/crm_attribute -n hana_rh2_glob_srHook -v SFAIL -t crm_config -s SAPHanaSR> rc=256
    [57670]{-1}[-1/-1] 2021-09-01 11:54:43.877943 i ha_dr_SAPHanaSR  SAPHanaSR.py(00112) : SAPHanaSR CALLING CRM: <sudo /usr/sbin/crm_attribute -n hana_rh2_glob_srHook -v SOK -t crm_config -s SAPHanaSR> rc=256
   

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.