Enabling and Managing Consistency Groups in OpenShift: A Step-by-Step Guide

Updated

This article provides a comprehensive guide to enabling and using Consistency Groups (CG) with OpenShift 4.18 and above. The CG feature allows for grouped snapshot operations, and data consistency across related volumes. To enable this feature, OpenShift’s TechPreviewNoUpgrade feature gate must be activated, which installs the necessary VolumeGroupSnapshot Custom Resource Definitions (CRDs). The document details step-by-step instructions for configuring CG within OpenShift, including modifying cluster settings, labeling StorageClasses, and annotating DRPlacementControl (DRPC) resources. Additionally, it explains how to monitor replication activities via ReplicationGroupSource and ReplicationGroupDestination. This feature is in developer preview and requires manual configuration via CLI, as it is not yet supported through the OpenShift UI.

Tested versions of OpenShift are 4.18 and 4.19

Using Consistency Groups with OpenShift

Enabling Volume Group Snapshot Feature

To enable Consistency Groups, the OpenShift feature gates must be configured to install the necessary VolumeGroupSnapshot, VolumeGroupSnapshotClasses, and VolumeGroupSnapshotContents CRDs. This requires activating the TechPreviewNoUpgrade feature gate.

Follow these steps:

Note: This procedure is also documented in the OpenShift documentation.

  1. Edit the cluster FeatureGate CR:

    $ oc edit featuregate cluster

  2. Enable TechPreviewNoUpgrade:

    spec:
  featureSet: TechPreviewNoUpgrade

  3. Verify that the required CRDs are installed:

    $ oc get crd | grep volumegroupsnapshot
volumegroupsnapshotclasses.groupsnapshot.storage.k8s.io 2024-10-10T10:48:51Z
volumegroupsnapshotcontents.groupsnapshot.storage.k8s.io 2024-10-10T10:48:52Z
volumegroupsnapshots.groupsnapshot.storage.k8s.io 2024-10-10T10:48:52Z

    Wait until all three CRDs are installed.

  4. Ensure StorageClasses have the required label:

    labels:
  ramendr.openshift.io/storageid: 2061a07f-60f0-4be1-a82d-697929f7c7e8

Note: CG in OpenShift 4.18 is a developer preview feature and is not supported via the UI.

Enabling Consistency Groups for Disaster Recovery (DR)

When DR is enabled, workloads are managed via DRPlacementControl (DRPC) resources on the Hub cluster. To enable CG for a workload, follow these steps:

Steps to Enable CG in DRPC

  1. Deploy the application and ensure all pods are in a running state.

  2. Scale down Ramen on the Hub before assigning DRPolicy:

    $ oc scale deploy -n openshift-operators ramen-hub-operator --replicas=0

  3. Assign DRPolicy to the workload.

  4. Edit the DRPC resource:

    oc edit drpc -n <namespace> <name>

    Add the following annotation:

    metadata:
  annotations:
    drplacementcontrol.ramendr.openshift.io/is-cg-enabled: "true"

    Example:

    apiVersion: ramendr.openshift.io/v1alpha1
kind: DRPlacementControl
metadata:
  annotations:
    drplacementcontrol.ramendr.openshift.io/is-cg-enabled: "true"

  5. Scale Ramen back up after editing DRPC:

    oc scale deploy -n openshift-operators ramen-hub-operator --replicas=1

Monitoring Consistency Groups

ReplicationGroupSource (Primary Cluster)

ReplicationGroupSource functions similarly to VolSync’s ReplicationSource, managing Volume Groups as a source. Check for active ReplicationGroupSource instances:

oc get replicationgroupsource -A

Example output:

NAMESPACE        NAME                                                              LAST SYNC              DURATION        NEXT SYNC              SOURCE                                                                                                 LAST SYNC START
busybox-sub-cg   busybox-sub-cg-placement-1-drpccephfs-899ee5aa59525d8268ca5c346   2025-03-28T21:35:27Z   27.943177116s   2025-03-28T21:40:00Z   {"matchLabels":{"ramendr.openshift.io/consistency-group":"cephfs-899ee5aa59525d8268ca5c346c61b678"}}

Each ReplicationGroupSource manages associated ReplicationSource instances:

oc get replicationsource -A

Example:

NAMESPACE        NAME               SOURCE                   LAST SYNC              DURATION        NEXT SYNC
busybox-sub-cg   busybox-cg1-pvc1   vs-cg-busybox-cg1-pvc1   2025-03-28T21:35:27Z   21.933815449s   
busybox-sub-cg   busybox-cg1-pvc2   vs-cg-busybox-cg1-pvc2   2025-03-28T21:35:25Z   19.840215159s

ReplicationGroupDestination (Secondary Cluster)

ReplicationGroupDestination manages Volume Groups as a destination on the secondary cluster. Check for active instances:

oc get replicationgroupdestination -A

Example output:

NAMESPACE        NAME                                                              LAST SYNC              DURATION          LAST SYNC START
busybox-sub-cg   busybox-sub-cg-placement-1-drpccephfs-490b2f95cb6548efabaf18237   2025-03-28T21:35:28Z   4m57.564135266s   2025-03-28T21:35:28Z

Each ReplicationGroupDestination manages associated ReplicationDestination instances:

oc get replicationdestination -A

Example:

NAMESPACE        NAME               LAST SYNC              DURATION          NEXT SYNC
busybox-sub-cg   busybox-cg1-pvc1   2025-03-28T21:35:27Z   4m56.801828752s   
busybox-sub-cg   busybox-cg1-pvc2   2025-03-28T21:35:25Z   4m53.810516668s

Summary

This guide provides the necessary steps to enable and manage Consistency Groups (CGs) in OpenShift 4.18 and above for CephFS. Using DRPlacementControl and VolumeGroupReplication, CGs ensures workload consistency across clusters, enhancing data protection and recovery capabilities.

Article Type