Storage optimization using Ceph Autoscaler and a Target Size Ratio for PG Increase/Placement on Large Pools - OpenShift Data Foundation (ODF) v4.19+
General Availability Steps in ODF v4.19+ Clusters:
Chapter 22. Configuring Ceph target size ratios
Dev Preview Steps in ODF v4.18 and below Clusters:
Note: Developer preview releases are meant for customers who are willing to evaluate new products or releases of products in an early stage of product development.
This is a live document only to be referenced for feature evaluation. This is unsupported for the production clusters. If you find any mistakes or missing instructions or need any assistance, please add an Issue or write to us @ ocs-devpreview@redhat.com.
Introduction
Under the assumption that the majority of the workload consumes a single storage service which is either Block, File, or Object, it's possible to optimize the OpenShift Data Foundation behavior by setting the target size ratio for any pool.
First, we will need to determine the desired ratio for storage usage
Second, we will need to follow the following two steps and set the right value.
- Disable reconciliation for the particular pool type
- Edit the CR for the pool to a new target size ratio value
Determine the Desired Ratio
To optimize storage depending on the cluster usage, determine how you expect to fill the cluster between the three types of storage: Block, Shared Filesystem, and Object storage. A relative value of expected usage will then need to be assigned to each of the three storage types. The value is the “targetSizeRatio”.
For simplicity, the values can be chosen to add up to 1.0. Each value should also be non-zero if very small.
Requiring a small nonzero number is a defect - A value of zero is valid - See Jira This content is not included.DFBUGS-2657.
For unused Pools which should be 0, use 0.001 until the defect (Jira) is resolved.
Example 1: Optimize for 80% in block storage and 10% in each of the Filesystem and Object store
Block pool targetSizeRatio: 0.8
Filesystem targetSizeRatio: 0.1
Object store targetSizeRatio: 0.1
Example 2: Optimize for 98% of storage in the filesystem:
Block pool targetSizeRatio: 0.01
Filesystem targetSizeRatio: 0.98
Object store targetSizeRatio: 0.01
Configuration
To override the default ratios set by ODF, disable the reconciliation in Step 1, then set the desired ratios in Step 2.
Disable reconcile for the particular pool type
Patch the StorageCluster to disable reconciliation for the desired pool. Otherwise, the change we make later would be immediately reverted by the reconciliation of the CR.
Disable Reconcile for CephBlockPools
# oc patch storagecluster ocs-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/managedResources/cephBlockPools/reconcileStrategy", "value": ignore }]'
Disable Reconcile for CephFilesystems
# oc patch storagecluster ocs-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/managedResources/cephFilesystems/reconcileStrategy", "value": ignore }]'
Disable Reconcile for CephObjectStores
# oc patch storagecluster ocs-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/managedResources/cephObjectStores/reconcileStrategy", "value": ignore }]'
Configure the new target size
Edit the CR for the pool to a new target size ratio value. This can be achieved by either doing oc edit the CR or doing an exact oc patch.
Patch for Cephblockpool
# oc patch cephblockpool ocs-storagecluster-cephblockpool -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/replicated/targetSizeRatio", "value": <0-1> }]'
Patch for Cephfilesystem
# oc patch cephfilesystem ocs-storagecluster-cephfilesystem -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/dataPools/0/replicated/targetSizeRatio", "value": <0-1> }]'
Patch for Cephobjectstore
Note: Object configuration is relevant only for bare metal or vSphere hosting environments.
# oc patch cephobjectstore ocs-storagecluster-cephobjectstore -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/dataPool/replicated/targetSizeRatio", "value": <0-1> }]'