OSD Weight and Pod Affinity configuration in OpenShift Container Storage - Developer Preview

Issue

In some instances, it might be required to assign an object storage device (OSD) to all the solid state drives (SSDs) on a host machine including a partitioned device that has a dedicated partition for the operating system. In such cases, you must reduce the load on this OSD as it shares the same physical device as the local operating system.

Resolution

Note: A support exception is required for running this solution in production because a wrong configuration can break the cluster.
Developer preview releases are meant for customers who are willing to evaluate new products or releases of products in an early stage of product development. In this case, OCS 4.8 does not support configuring OSD weight and pod affinity in the production environment.

This is a live document to be used in various environments and configurations. If you find any mistakes or missing instructions or need any assistance, please add an Content from github.com is not included.Issue or write us @ ocs-devpreview@redhat.com.

To reduce the load on the OSD, you need to configure either one or both of the following options depending on your cluster setup and workload:

• InitialWeight - controls the capacity for data placement
• PrimaryAffinity - Sets the primary OSD for the placement group that serves the read load

Diagnostic Steps

Setting the initialWeight value to reduce the write load

To reduce the load, set an explicit value for initial weight, which is lower than the derived value from the device capacity. By default, a weight value is assigned to each OSD based on the capacity in TB units. For example, an OSD over a 2 TB device has a weight of 2 and over a 512 GB device has a weight of 0.5. You can set a value for the optional field, initialWeight of storageDeviceSets in StorageCluster CR as shown in the following example.

Example to set initialWeight:

[...]
storageDeviceSets:
  - name: example-deviceset1
    count: 3
    initialWeight: 0.5TiB
    dataPVCTemplate:
      spec:
        storageClassName: gp2
        accessModes:
        - ReadWriteOnce
        volumeMode: Block
        resources:
          requests:
            storage: 2TiB
[...]

Setting the primaryAffinity value to reduce the read load

You can set a lower value for primary affinity which is a real number within the range of [0, 1] and the default value is 1. Note that when you set primary affinity value to 0, OSD will not be used as primary. The only exception is when Ceph has to use it during failures.

You can set a value to the optional field, primaryAffinityof storageDeviceSets in StorageCluster CR as shown in the following example.

Example to set primaryAffinitiy:

[...]
storageDeviceSets:
  - name: example-deviceset1
    count: 3
    resources: {}
    placement: {}
    primaryAffinity: 0.4
[...]

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.