Exploring RHEL High Availability's Components - sbd and fence_sbd

Updated

Contents

Overview

Applicable Environments

  • Red Hat Enterprise Linux (RHEL) 6, 7, 8, or 9 with the High Availability Add-On
    • At least RHEL 6 Update 8, RHEL 7 Update 1, or later update levels of either
  • pacemaker in use

Useful References and Guides

Introduction

sbd and fence_sbd are components of the RHEL High Availability Add-On which provide several methods of fencing a node.

This guide aims to explore the implementation, features, behaviors, and technical details of these components involved in sbd fencing. It is intended for organizations looking for a deeper understanding of these components so they can take full advantage of their capabilities and optimize their deployment in a cluster.

For readers looking for a simpler approach, check out the above references that dive more directly into examples and practical guidance for setting these components up. These below sections attempt to give more insight into the implementation and operation of the component, but are not meant to be procedural instructions - however the other references above attempt to offer guidance on that side as well.

Concepts

Summary of how the pieces fit together

The sbd utility can be used to initiate different actions or spawn health and quorum watchers or disk watchers.

The systemd sbd.service unit executes the sbd utility in "watch" mode - initiating the configured watchers.

The sbd inquisitor initializes the watchdog timer device with a countdown that must be frequently refreshed to prevent the system from resetting. The inquisitor monitors the watcher statuses, and refreshes the watchdog timer device if all is healthy.

Health and quorum watchers or disk watchers signal if they detect a problem in their watched area.

Disk watchers poll their assigned device for messages. Doing this, they detect issues accessing the devices as well.

The sbd utility can be executed from any node with a "message" command to send an instruction to a disk watcher on another node.

The fence_sbd fence-agent can be called with "reboot" or "off" actions for a node, which cause it to write such an instruction to disks that are being watched by other nodes.

The disk watcher on that node can see this message and follow the given instruction.

If the cluster has stonith-watchdog-timeout configured, any time a member leaves the membership uncleanly the cluster can assume after that length of time that either the health and quorum watcher or the disk watcher has signaled the problem to its inquisitor and thus caused the node to reboot itself or the (hardware-)watchdog has taken down the node. Thus, other cluster nodes can assume that node has self-fenced if stonith-watchdog-timeout has passed.

What is fencing?

Fencing is the means by which a cluster controls access to shared resources throughout the cluster. If a node loses contact with other cluster members, becomes unresponsive, or is misbehaving in some way, the cluster may need to prevent that node from accessing those shared resources.

What is self-fencing?

Self-fencing describes a STONITH method in which a node controls its own state or access level related to shared resources.

This is in contrast with other typical STONITH methods in which the authoritative portion of a cluster interacts with some external mechanism to control the state or access level of another member.

To achieve self-fencing, there must be some way for a node to recognize when it has been declared problematic by the cluster, so it can take action against itself. The cluster must also have some way to be sure that the problematic node has or should have fenced itself.

sbd provides a method of self-fencing.

What is a watchdog timer (WDT)?

A watchdog timer (abbreviated "WDT") device is a component integrated into a system's hardware platform; this device is able execute a system reset if software running within the operating system does not interact with the device in an expected amount of time.

The linux kernel provides a watchdog subsystem and API for these devices, along with various drivers to handle specific hardware models. The kernel watchdog subsystem and drivers present an interface to watchdog timer devices in the form of device nodes - typically /dev/watchdog or one or more /dev/watchdog[0-9]* devices.

If some application or service wishes to activate watchdog functionality, it opens this device node; this triggers the kernel to activate the hardware watchdog timer's countdown. Once the device file has been opened and the countdown timer starts counting down, it is up to the application to write data to the device again to reset the countdown timer. If that countdown timer ever expires, then the watchdog device will reset the system causing it to reboot.

The watchdog API allows applications to issue instructions to control things like how long the timeout should be set to, or what the behavior should be if the device file is closed by the application.

This is what makes self-fencing reliable. A node is either able to determine its own state and the implied need for self-fencing, or it fails to periodically trigger the watchdog device and is thus reliably taken down as well. Even if the need for self-fencing is detected properly but issues happening on actually doing so would, for example, delay taking down the node, the hardware watchdog still assures that it will be down within a defined time.

What does sbd do?

sbd is a daemon that can be deployed in a RHEL High Availability cluster to facilitate fencing through several methods. Its responsibilities include:

  • Monitor the health of a node within the cluster in various ways.
  • Monitor up to three storage devices that are shared throughout the cluster for any instructions sent by other nodes to this node, resetting this node if such an instruction is received.
  • Interact with the system's watchdog device to initiate its countdown timer and keep resetting it as long as a node is healthy in the cluster.
  • If a problem is detected with this node's health or status in the cluster, attempt to reboot the operating system as fencing to prevent it from further accessing resources shared by the cluster.

What does fence_sbd do?

fence_sbd is a STONITH fence-agent that can initiate a "poison-pill" message to another node, instructing that node to reset itself. fence_sbd must be given access to 1-3 block devices shared throughout the cluster, over which it sends such poison-pill messages.

This agent can be integrated into the STONITH configuration of the cluster so that if the members decide a node is problematic or is no longer a valid member, they can trigger this agent to send a message to the other node.

What does cluster property stonith-watchdog-timeout do?

When sbd is running on all nodes of a cluster, that daemon gives an assurance that any non-quorate or unhealthy node will be rebooted within the stonith-watchdog-timeout period.

Pacemaker offers a cluster property called stonith-watchdog-timeout that treats any pending STONITH action as having succeeded after a member has been missing for that amount of time. This stonith-watchdog-timeout value should be longer than the sbd watchdog-device's timer, and thus as long as the cluster waits that long, it can be sure any missing node has fenced itself via sbd.

stonith-watchdog-timeout causes the stonith-ng daemon to essentially have a hidden STONITH device. This "device" simply reports that a STONITH action of a member is successful after the timeout has passed.

On RHEL 8.4 or later, this hidden STONITH method enabled via stonith-watchdog-timeout can optionally be turned into a visible fencing device; you set up a fence_watchdog-based device as you would any other fencing device. Being visible, this fencing device can be configured to work in conjunction with other administrator-configured STONITH devices. Using a fencing topology it can, for example, be configured to serve as a fallback fencing method if another administrator-configured STONITH device fails. (If the reason that other STONITH device failed was a timeout you need RHEL 9.3 / 8.9 or current z-streams for anything from RHEL 8.4 for the topology timeout - stonith-watchdog-timeout + timeout of that other fencing-device - to be derived correctly.) Using the instance attribute pcmk_host_list for the fence_watchdog-based device, it is possible to operate a cluster where it is not possible or desired to run sbd-daemon on all nodes (for example, because they are remote nodes or do not provide a usable or reliable hardware watchdog).

Note: On RHEL 7, stonith-watchdog-timeout cannot be used as a failback fencing method if another administrator-configured STONITH device fails.

How does a RHEL High Availability cluster achieve fencing with sbd?

There are several ways in which the features of sbd, fence_sbd, and the other RHEL High Availability components can combine to result in a self-fencing arrangement. In some environments one or the other feature may be configured, or in some cases they may all be deployed together to offer fencing capabilities in as many scenarios as possible.

The three ways in which fencing with sbd can be achieved are:

  • sbd health and quorum monitoring: A server self-fences if it detects its in an unhealthy state.

    • The sbd daemon runs on nodes of the cluster and monitors the health and quorum status of each node in a few ways.
    • If a problem is detected on that node in the area of its health or quorum, sbd stops updating the watchdog timer device, causing that system to reboot once the countdown expires.

  • sbd stonith-watchdog-timeout fencing - The cluster assumes a missing node will have self-fenced after the watchdog countdown passes.

    • sbd health and quorum monitoring being enabled on the nodes of the cluster results in assurances about what will lead to a node self-fencing.
    • With a node set to self-fence via sbd if a health or quorum problem is detected, the cluster can then assume any node which is no longer a member has successfully watchdog-fenced itself after a certain length of time has passed.
    • Quorum policies ensure that only one partition in the cluster can ever be quorate - so if the local node does not see another node as a member of the local partition, then that other node must consider itself inquorate.
    • A node that considers itself inquorate will stop updating its watchdog device via sbd - and thus will be fenced within a defined length of time after the membership change.
    • So, as long as the cluster waits longer than the countdown timer following a membership change, it is safe to assume any missing member is successfully self-fenced.

  • sbd poison-pill fencing via block-device - (which automatically enables the previously mentioned sbd health and quorum monitoring method, in this case either quorate as passed from pacemaker/corosync or having access to a quorate number of block devices): sbd listens for messages on a shared block device, and a special fence agent is able to send fencing instructions to it via that block device.

    • Each node running sbd has access to 1-3 shared storage devices for sbd. sbd watches an area of those devices for messages to this node.
    • The cluster is configured to have a fence_sbd STONITH device associated with some or all members.
    • If one of those members becomes problematic to the cluster, fence_sbd is initiated.
    • fence_sbd writes a message to the sbd storage devices.
    • If the target node is still alive and responsive, it will receive this message and respond that it is self-fencing, allowing fencing to succeed quickly. If the target node does not respond, the initiating node will wait a certain amount of time (msgwait) before declaring the message-sending a success. By that time, the node will have either received its poison-pill message, will have self-fenced due to health and quorum monitoring, or will have been taken down by the (hardware-)watchdog.

Typical deployments usually have either watchdog fencing or poison-pill fencing configured. Some kind of health and quorum monitoring will be done implicitly in both cases. Configuring both watchdog fencing and poison-pill fencing for the same target node is dangerous and should be prevented by newer pcs versions. Sbd-daemon running with disks will allow a node to survive either by disk health or quorum and thus a node losing quorum will not self-fence. Thus watchdog fencing would not work and this would impose a risk of split-brain.

sbd Component Explained

sbd commands

The /usr/sbin/sbd binary serves both as a utility to initiate certain actions, as well as the inquisitor and watcher daemon processes. In other words: sbd can be called with one of various commands, and that command may carry out a function and exit, or it may spawn an inquisitor or watcher process that continues to run on its own. sbd accepts the following commands (all except 'watch' require RHEL 7 Update 4 or later):

  • watch: Spawn an inquisitor which will then spawn watchers to monitor the health status of this node, or monitor a disk for incoming messages.
  • create: Initialize a block device to be monitored by a disk watcher and have instructions like poison-pill messages sent over. This create command sets up a header with configuration details for sbd and sets up "slots" that individual nodes can allocate as the space to communicate instructions to them.
  • dump: Show the sbd header of an already-initialized block-device, in human-readable form.
  • list: Show the allocated node-slots of an initialized block device and the status of each node ("clear" or a state indicating a node has been fenced).
  • allocate: (Shouldn't be needed in most environments): Allocates a slot for a particular node on a block device.
  • message: Send an instruction to specified node over its allocated slot on specified block device(s). This command is used by fence_sbd to send poison-pill messages.
    • With multiple block-devices specified, this command will only succeed if sending and receiving a reply succeeds over a majority of devices. With 1 device, that 1 must send and receive the reply successfully. With 2, at least 2 must succeed. With 3, at least 2 must succeed. If less than a majority succeeds, this sbd message command is considered to have failed.
    • Completion time of a message command is restricted by the msgwait timeout that is stored on the block device(s) during initialization by the administrator. This timeout controls the maximum amount of time sbd message will wait for a reply before giving up and declaring a failure. The length of time that a block-device may be acceptably unresponsive should dictate the definition of this setting.
  • test-watchdog: Arm the watchdog (optionally configured with -w option) which will cause the system to reset if the watchdog is working. If issued from an interactive session, it will prompt for confirmation.
  • query-watchdog: Check for available watchdog devices and print some information.

Warning: This command will arm the watchdog during query, and if your watchdog refuses disarming (for example, if its kernel module has the 'nowayout' parameter set) this will reset your system.

Initialization of sbd daemon

When the sbd.service systemd service is started, it calls sbd with a command of "watch". This causes sbd to initialize an "inquisitor" process that will be in charge of carrying out sbd's primary work. The original sbd process that was started by systemd spawns this inquisitor, then waits for a signal from it indicating it has observed conditions that qualify the node to be healthy. If the inquisitor indicates the node is healthy, the sbd process exits and leaves the inquisitor to perform the ongoing work.

This service is a bit different from the typical others managed by systemd, in that an administrator does not start and stop it directly. Rather, it is designed such that it can only be started by its dependent service corosync. So when corosync starts - possibly via pcs cluster start, or by it starting automatically on boot - it also causes sbd to start.

sbd inquisitor

The inquisitor is the main coordinating process for sbd's cluster-monitoring workers - it spawns other "watchers" to do the monitoring, then listens for signals received back from those watchers that indicate their health.

The timing and type of the signals sent by watchers gives the inquisitor what it needs to know to decide if each is healthy, and the combined results of all of those watchers allows the inquisitor to decide what action should be taken.

The inquisitor process can be seen clearly in the process listing by its name:

# ps aux | grep -e COMMAND -e "sbd: inquisitor" | grep -v grep                                                                                                                               
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     30156  0.0  0.8 116856 16624 ?        SL   12:02   0:00 sbd: inquisitor

Once the inquisitor launches the watcher processes, it will watch for the first indication that the cluster is up and ready on this node, then will initialize the watchdog device with the countdown timer.

After the countdown timer is "armed", sbd must continue to update that watchdog device within the timer period to avoid the system being reset. The inquisitor takes the following conditions as an indication the cluster is healthy on this node and the watchdog should be initially armed:

  • sbd is not configured to watch any block devices for poison-pill messages, and one of the pacemaker or cluster watchers is declared healthy.
  • sbd is configured to watch block devices, and a majority of those device's watchers are healthy - indicating those devices have been initialized and can be read from successfully.

Once the watchdog timer is initialized, the inquisitor will monitor the status of its watchers to decide if it should continue updating the watchdog timer. It chooses to update that timer on the following conditions:

  • All watchers are healthy - cluster, pacemaker, and disk
  • sbd is not configured to watch disks, and cluster and pacemaker watchers are healthy
  • sbd is not configured to watch disks, and one of cluster or pacemaker is healthy while the other has not yet been seen healthy.
  • sbd is configured to watch disks, the cluster or pacemaker watchers have not yet been seen as healthy, but a majority of disk watchers are healthy.

sbd disk watchers

sbd can be configured to watch up to three block devices for poison-pill messages. Each device will have a watcher process spawned for it, and that watcher process can be seen in the process listing:

# ps aux | grep -e COMMAND -e "sbd: watcher: /dev" | grep -v grep
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      8714  0.0  0.8 116956 16704 ?        SL   08:53   0:03 sbd: watcher: /dev/disk/by-id/dm-uuid-mpath-360014054540864241b04b67af8351a38 - slot: 1 - uuid: 81b00312-11e8-4e34-b6db-850cd3c0dcec
root      8715  0.0  0.8 116956 16704 ?        SL   08:53   0:02 sbd: watcher: /dev/disk/by-id/dm-uuid-mpath-360014058b858513c3044413ace481447 - slot: 1 - uuid: 3f1eea41-6a5d-4d6c-91c1-3cfadcd0154d
root      8716  0.0  0.8 116956 16704 ?        SL   08:53   0:03 sbd: watcher: /dev/disk/by-id/dm-uuid-mpath-360014059b3b71fdd6254d519c085050c - slot: 1 - uuid: 1e492896-ffd7-4f3b-b21d-b091713e1b65

Each disk should be initialized by the administrator before sbd can use it, using pcs stonith sbd device setup. This in turn calls sbd create to set up a metadata structure on the device, including the chosen or default timeout values that sbd abides by.

When the disk watcher starts, it attempts to either find an existing slot or allocate a new slot on the device in which messages can be written - there are up to 255 slots. This watcher then will periodically monitor that slot for messages, and take action in response to any that are received.

These messages can deliver any of the following instructions: test (respond to the message with a test reply), reset (initiate a SysRq + 'b' hard reboot), off (initiate a SysRq + 'o' hard-shutdown), or crashdump (initiate a SysRq + 'c' kernel panic). These messages can come from administrator-initiated commands throughout the cluster issued via sbd or pcs stonith sbd device message, or can come from a fence_sbd STONITH device configured in the cluster and executed during a fence action against this node.

When the disk watcher receives an instruction, it responds to inform the source that the message has been received and then carries out the specified action. If this was a reset or an off instruction, then self-fencing is carried out by shutting down the system using the SysRq facility. The response sent back to the source prior to shutting down allows the node that initiated that message - usually via fence_sbd - to confirm this node is self-fencing and return a successful result from that action.

While the disk watcher is monitoring its block device, it tracks the timing and latency of its interactions and compares this against the configured timeouts. If timeouts are exceeded, then the watcher may report warnings in the logs to notify the administrator of the problem. If the disk watcher is delayed long enough in its interactions with the device, the inquisitor may consider it "outdated" and weigh that into its decision of whether the watchdog device should continue being updated.

sbd health and quorum watchers

When sbd is started in a pacemaker environment, it enables watchers to monitor the cluster stack for health status. These watchers can be seen in the process listing by names indicating which part of the stack they are monitoring:

# ps aux | grep -e COMMAND -e "sbd: watcher: Pacemaker" -e "sbd: watcher: Cluster" | grep -v grep
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     30158  0.0  1.1 121156 20792 ?        SL   12:02   0:07 sbd: watcher: Pacemaker
root     30159  0.0  1.3 125912 24912 ?        SL   12:02   0:01 sbd: watcher: Cluster

These watchers have the following responsibilities:

  • sbd: watcher: Cluster: The cluster watcher is responsible for monitoring the status of corosync and the cluster membership. It does this by connecting to the sbd:cluster CPG group and registering a callback to receive configuration changes in that CPG. Quorum information from corosync is collected and evaluated via the Pacemaker watcher. In a 2-node cluster (not a cluster with 2 cluster nodes and a qdevice), though, a node is going to stay quorate even if the second node gets lost. Thus this configuration is not usable for watchdog fencing. For poison-pill fencing with Pacemaker awareness (needed for a single disk not to become a single point of failure) in a 2-node cluster this watcher signals health only if both nodes are registered with the CPG protocol. Larger clusters or ones that are using qdevice are considered healthy by this watcher if only one node (presumably the local one) is registered with corosync. Liveness of the corosync daemon is checked by periodically polling votequorum-info (not blocked during sync-phase of corosync).

  • sbd: watcher: Pacemaker: The Pacemaker watcher attempts to detect whether the pacemaker system is healthy and responsive. It does so by connecting to the cib daemon and registering to receive configuration change notifications. The watcher waits a few seconds to receive such a notification, or will force a refresh of its copy of the CIB if no notifications come in within that time. As long as the CIB connection is producing configuration refresh notifications, this watcher reports healthy status. It also evaluates the node state via the cib-information received (quorum-state and if the state of the resources is considered unclean on the node).

Notable files provided by the sbd package

  • /etc/sysconfig/sbd - Primary configuration file for sbd. Settings should ideally be configured through pcs stonith sbd enable [...].
  • /usr/lib/systemd/system/sbd.service - systemd service for sbd. Can not be started/stopped directly, but rather only by enabling sbd in systemd and then starting corosync (usually via pcs cluster start).
  • /usr/sbin/sbd - sbd daemon. Should not need to be executed by administrators in typical deployments.

fence-agents-sbd Component Explained

fence_sbd structure and operation

The fence_sbd agent is pretty straightforward, with an implementation based on the standard fence-agents-common fencing library used by most other fence-agents that Red Hat supplies.

With a STONITH device configured to use this agent, when the cluster decides a node needs to be fenced, it executes the agent using the off or reboot action (depending on configuration and circumstances). The cluster may also need to call fence_sbd with a list action in order to determine which nodes can be fenced.

fence_sbd execution of off, reboot, and list actions

When this agent is executed with an off, reboot, or list action, it executes an sbd messagecommand with the corresponding instruction. See the above section onsbd commands` for more details of how this works.

A message by sbd of "off" or "reset" (reset being called by fence_sbd's "reboot" action) will attempt to send that instruction over the configured block device(s) and wait for a reply from the target node. As noted in the sbd commands section above - the message sending and reply receiving must succeed over a majority of devices in order for this to succeed. So if the nodes are only able to communicate over half or less of their devices, this fence_sbd operation will be considered a failure.

If the target of the fence action is still alive and responsive, sbd's disk watcher should be listening to that device on the other end, should follow the instruction included in the message sent over the devices, and should reply. If a majority of devices get that successful reply, then fence_sbd's operation succeeds.

Known Issue: Due to a flaw in fence_sbd, in a scenario with multiple block devices fence-actions fails if one of the block devices is unresponsive. This issue was fixed with the following errata:

Red Hat Enterprise Linux 9

Notable files provided by the fence-agents-sbd package

  • /usr/sbin/fence_sbd: Fence agent to be configured as a STONITH device, which then sends a poison-pill message to other nodes via shared block device(s).

Configuration of sbd components

Configuration of sbd daemon

sbd's settings are read by the daemon via environment variables configured in /etc/sysconfig/sbd - and can be configured at the time of pcs stonith sbd enable. The available settings to configure are:

  • SBD_DELAY_START (integer | string: no): How long sbd will delay its initialization on startup before interacting with the watchdog or initiating the inquisitor or watchers. Gives time for devices to fully come online and the cluster to stabilize before performing monitoring.

  • SBD_DEVICE (string: semi-colon-separated list of block-device paths): Up to three storage devices shared throughout the cluster, which sbd should monitor and watch for messages over.

  • SBD_MOVE_TO_ROOT_CGROUP (string: yes|no|auto): If CPU Accounting is enabled, the default is not to assign any RT-budget to system.slice, which prevents sbd from running RR scheduling. If SBD_MOVE_TO_ROOT_CGROUP is set to yes, sbd processes are moved from the slice they were originally started on to the root slice. If set to auto, sbd processes stay in the original slice if the slice has RT-budget assigned, otherwise the processes are moved to the root slice. In the course of switching to cgroup-v2 in RHEL 9, kernel configuration was altered so that extra budget for realtime-execution no longer needs to be configured.

  • SBD_OPTS (string): stdin / command-line options to pass to the sbd process when starting it. See sbd -h or sbd(8) for details.

  • SBD_PACEMAKER (string: yes | no): Whether sbd is running in a Pacemaker environment, and thus should start its cluster and Pacemaker watchers. Should typically be yes in RHEL High Availability clusters. Equivalent to SBD_OPTS="-P -c".

  • SBD_STARTMODE (string: always | clean): When sbd is configured to monitor block devices for poison-pill messages, this setting dictates the startup behavior of sbd the next time it starts up after a poison-pill fence action against this node. If always, then sbd will proceed to starting normally whether this follows poison-pill fence action or not. If clean, sbd will only start if there is no record of a poison-pill message directed at this node - with the goal being to prevent a node from rejoining the cluster immediately after it had a problem causing fencing.

  • SBD_TIMEOUT_ACTION(string: comma-separated combination ofnoflush|flush plus reboot|crashdump|off): Actions to be executed when the watchers do not report to the sbd master within expected timeouts, or one of the watchers detects that the master process has died. Defaults to “flush, boot”; if just one of the actions is given, the other remains at its default value. This setting does not affect actions such as off, crashdump, and reboot` explicitly triggered by means of message slots, and it does not configure the action a watchdog would trigger.

  • SBD_WATCHDOG_DEV (string): Path to the watchdog device that should be used.

  • SBD_WATCHDOG_TIMEOUT (integer): Value to set the watchdog device's countdown timer to. This dictates the maximum amount of time it may take for a node to reset if it becomes unhealthy or unresponsive.

Default sbd daemon settings deployed by pcs stonith sbd enable

pcs allows specifying values for these various settings, so the defaults are as follows unless overridden by the administrator:

  • SBD_DELAY_START: "no" (aka 0, or no delay).

  • SBD_DEVICE: (empty - meaning do not utilize block-device messaging)

  • SBD_OPTS: "-n <nodename>"

  • SBD_PACEMAKER: "yes" (enables Pacemaker and cluster watchers)

  • SBD_STARTMODE: "always" (do not block startup of sbd even if the node was just fenced via poison-pill).

  • SBD_WATCHDOG_DEV: "/dev/watchdog"

  • SBD_WATCHDOG_TIMEOUT: "10" (seconds)

How sbd health and quorum monitoring is configured

If the systemd sbd.service unit is enabled, the sbd command will automatically be started in "watch" mode upon corosync.service starting.

sbd.service can be enabled via pcs stonith sbd enable.

SBD_WATCHDOG_DEV and SBD_WATCHDOG_TIMEOUT influence how sbd interacts with the watchdog device in sbd health and quorum monitoring.

How sbd stonith-watchdog-timeout fencing is configured

Pacemaker offers the stonith-watchdog-timeout cluster property. This value enables the cluster to operate with sbd stonith-watchdog-timeout fencing.

How a node is configured to receive sbd poison-pill fencing via fence_sbd

sbd's block devices are initialized for messaging-use with the appropriate on-disk structure by the sbd create command. This command is called by pcs stonith sbd device setup command.

sbd is enabled to listen on these block-devices by way of SBD_DEVICE. Watchers for up to three devices listed are initiated when sbd executes in watch mode.

Any node watching the shared devices is enabled to receive fence actions against it through sbd poison-pill fencing via fence_sbd.

How the cluster is configured to fence nodes through sbd poison-pill fencing via fence_sbd

The cluster is enabled to fence nodes via disk-messages by configuring a STONITH device using the fence_sbd agent and having that device associated with a node.

Nodes that are listening on the block device(s) should automatically be detected as associated with a device, so no configuration is necessary to activate that. (In other words: neither pcmk_host_list nor pcmk_host_map is required).

The fence_sbd STONITH device type is configured similarly to other fence-agents - the administrator creates a device in the cluster CIB using pcs stonith create:

# # Example:
# pcs stonith create sbd fence_sbd devices="/dev/disk/by-id/dm-mpath-uuid-1234567890"

Attributes can be supplied in the STONITH device configuration, with devices typically being the only one necessary.

An entire cluster can typically use a single shared fence_sbd device.

SBR
Product(s)
Category
Components
Tags
Article Type