ODF-4.14 | In-transit compression Developer Preview
Important: A developer preview feature is subject to Developer preview support limitations. Developer preview features are not intended to be run in production environments. The clusters deployed with the developer preview features are considered to be development clusters and are not supported through the Red Hat Customer Portal case management system. Development Preview features are meant for customers who are willing to evaluate new products or releases of products in an early stage of product development. If you need assistance with developer preview features, reach out to the ocs-devpreview@redhat.com mailing list and a member of the Red Hat Development Team will assist you as quickly as possible based on availability and work schedules. To know more about the support scope refer to the This content is not included.KCS.
What is In-Transit compression
On the wire compression of data can help a lot In multi-availability zones deployment by lowering latency & network costs. It can also help in cases where the Network Bandwidth is a bottleneck for performance.
We use the ceph msgrv2 protocol for achieving in-transit compression of data.
Important
Applying compression prior to encryption has security implications as it reduces the level of security of messages between peers.
Applying compression after encryption is not efficient and the cost reduction would be very minimal.
So In case both encryption and compression are enabled in-transit, compression setting will be ignored by ceph and message will not be compressed.
ODF Internal Mode
Enable compression during creation of storagecluster
While creating storagecluster include compression enabled true in network connections spec
network:
connections:
compression:
enabled: true
Wait till the storagecluster becomes ready
~ $ oc get storagecluster
NAME AGE PHASE EXTERNAL CREATED AT VERSION
ocs-storagecluster 62m Ready 2023-07-27T12:15:47Z 4.14.0
Enable compression on an existing storagecluster
Apply the patch for making compression enabled true in storagecluster network connections spec
oc patch storagecluster ocs-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/network", "value": {"connections": {"compression": {"enabled": true}}} }]'
Wait for the storagecluster to get ready
~ $ oc get storagecluster
NAME AGE PHASE EXTERNAL CREATED AT VERSION
ocs-storagecluster 141m Ready 2023-01-19T11:56:18Z 4.12.0
All the ceph daemons(mon, mgr, mds, osd) will now restart one by one to account for the new network settings
Wait for around 5-10 mins for all these pod restarts to happen. Once they are all done the cluster is ready to use.
rook-ceph-crashcollector-ip-10-0-152-121.ec2.internal-b7cbccx9z 1/1 Running 0 4m40s
rook-ceph-exporter-ip-10-0-152-121.ec2.internal-d6954cb79-kmh86 1/1 Running 0 4m40s
rook-ceph-exporter-ip-10-0-168-37.ec2.internal-76648694b4-vq6qf 1/1 Running 0 4m37s
rook-ceph-crashcollector-ip-10-0-168-37.ec2.internal-68655pbqv9 1/1 Running 0 4m37s
rook-ceph-exporter-ip-10-0-210-243.ec2.internal-68cfdf7769bf5jq 1/1 Running 0 4m35s
rook-ceph-crashcollector-ip-10-0-210-243.ec2.internal-686fpx29z 1/1 Running 0 4m35s
rook-ceph-mds-ocs-storagecluster-cephfilesystem-a-77885695kbqt7 2/2 Running 0 4m21s
rook-ceph-mon-b-84b685c9b9-lqnwf 2/2 Running 0 4m20s
rook-ceph-mds-ocs-storagecluster-cephfilesystem-b-6ffbc675d2c2z 2/2 Running 0 3m55s
rook-ceph-mon-c-5565dc4c59-6nnkd 2/2 Running 0 3m46s
rook-ceph-mon-a-5994c9ccff-mlhj5 2/2 Running 0 2m20s
rook-ceph-mgr-a-7cf656d8cf-8kxk8 2/2 Running 0 110s
rook-ceph-osd-0-7f6c74cf76-pffcl 2/2 Running 0 84s
rook-ceph-osd-1-7d8df4f48d-4vbtv 2/2 Running 0 60s
rook-ceph-osd-2-6dc5754456-gmg2r 2/2 Running 0 37s
Disable In-transit compression
Apply the patch on the storagecluster to mark compression enabled as false in network connections spec
oc patch storagecluster ocs-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/network/connections/compression", "value": {"enabled": false} }]'
Wait for the storagecluster to get ready
~ $ oc get storagecluster
NAME AGE PHASE EXTERNAL CREATED AT VERSION
ocs-storagecluster 80m Ready 2023-07-27T12:15:47Z 4.14.0
All the ceph daemons(mon, mgr, mds, osd) will now restart one by one to account for the new network settings
Wait for around 5-10 mins for all these pod restarts to happen. Once they are all done the cluster is ready to use.
rook-ceph-exporter-ip-10-0-210-243.ec2.internal-d76b776d-22lb6 1/1 Running 0 6m15s
rook-ceph-crashcollector-ip-10-0-210-243.ec2.internal-65b9dklgw 1/1 Running 0 6m15s
rook-ceph-exporter-ip-10-0-168-37.ec2.internal-674f486579-t8xj8 1/1 Running 0 6m12s
rook-ceph-crashcollector-ip-10-0-168-37.ec2.internal-66db7dz59s 1/1 Running 0 6m12s
rook-ceph-crashcollector-ip-10-0-152-121.ec2.internal-7d4fbggsq 1/1 Running 0 6m10s
rook-ceph-exporter-ip-10-0-152-121.ec2.internal-844b8b4746xk96b 1/1 Running 0 6m10s
rook-ceph-mds-ocs-storagecluster-cephfilesystem-a-87888f8842svh 2/2 Running 0 5m58s
rook-ceph-mon-a-759899c667-m5hvg 2/2 Running 0 5m54s
rook-ceph-mds-ocs-storagecluster-cephfilesystem-b-67cdc685bdk47 2/2 Running 0 5m32s
rook-ceph-mon-b-597794c4f5-7h8h6 2/2 Running 0 4m28s
rook-ceph-mon-c-67bc64b895-k9dj4 2/2 Running 0 3m5s
rook-ceph-mgr-a-85955b777d-4bxqt 2/2 Running 0 2m35s
rook-ceph-osd-0-5f6c58d547-zzzqr 2/2 Running 0 2m10s
rook-ceph-osd-1-7db89b5797-pjs7t 2/2 Running 0 106s
rook-ceph-osd-2-5d48b68595-hls8j 2/2 Running 0 83s
ODF External Mode
Enable compression on an existing storagecluster
Apply the patch to the storagecluster to enable compression
oc patch storagecluster ocs-external-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/network", "value": {"connections": {"compression": {"enabled": true}}} }]'
Wait for the storagecluster to get ready & all the storagecluster conditions to say successful/completed.
Login to to the RHCS cluster & Apply the in-transit compression ceph configurations
~ ceph config set global ms_osd_compress_mode force
~ ceph config set global rbd_default_map_options ms_mode=prefer-crc
Restart the ceph osd daemons
~ ceph orch ls
NAME PORTS RUNNING REFRESHED AGE PLACEMENT
mds.cephfs 1/2 3d ago 8d count:2
mgr 1/2 3d ago 9d label:mgr
mon 3/4 3d ago 9d label:mon
osd.all-available-devices 12 3d ago 8d *
~ ceph orch restart osd.all-available-devices
Wait for the osd restarts to be completed
Disable compression
Apply the patch on the storagecluster to disable compression
oc patch storagecluster ocs-external-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/network", "value": {"connections": {"compression": {"enabled": false}}} }]'
Wait for the storagecluster to get ready & all it's conditions to say successful/completed.
Login to the RHCS cluster & Remove the in-transit compression ceph configurations
~ ceph config rm global ms_osd_compress_mode
~ ceph config rm global rbd_default_map_options
Restart the ceph osd daemons
~ ceph orch ls
NAME PORTS RUNNING REFRESHED AGE PLACEMENT
mds.cephfs 1/2 3d ago 8d count:2
mgr 1/2 3d ago 9d label:mgr
mon 3/4 3d ago 9d label:mon
osd.all-available-devices 12 3d ago 8d *
~ ceph orch restart osd.all-available-devices
Wait for the osd restarts to be completed