[Ceph/ODF] Mount the ODF CephFS volume on an OpenShift Worker Node.

Environment

Red Hat OpenShift Container Platform (OCP) 4.x
Red Hat OpenShift Data Foundation (ODF) 4.x
Red Hat Ceph Storage (RHCS) 5
Red Hat Ceph Storage (RHCS) 6
Red Hat Ceph Storage (RHCS) 7
Red Hat Ceph Storage (RHCS) 8
Ceph File System (CephFS)

Issue

How to mount the ODF CephFS volume on an Openshift Worker Node.

Resolution

If not already in place, please use Configuring the Rook-Ceph Toolbox in OpenShift Data Foundation to access the Ceph CLI.

1. List your nodes to get the host name of the worker node you want to mount the CephFS volume on:

[OpenShift CLI]$ oc get nodes -o wide

2. Run the following command to gather the Ceph Mon v1 and v2 IP addresses:

[Ceph Cli]$ ceph mon stat
e3: 3 mons at {a=[v2:172.30.35.41:3300/0,v1:172.30.35.41:6789/0],b=[v2:172.30.138.26:3300/0,v1:172.30.138.26:6789/0],c=[v2:172.30.194.211:3300/0,v1:172.30.194.211:6789/0]} removed_ranks: {}, election epoch 12, leader 0 a, quorum 0,1,2 a,b,c

3. Run the following command to gather the client-admin-key:
   Take note of the key value in the output (e.g.AQBqjQ9kpG9oHxAATt1PV9gfdcdTkrZJSY/e/g==)

[Ceph Cli]$ ceph auth get client.admin
[client.admin]
	key = AQBqjQ9kpG9oHxAATt1PV9gfdcdTkrZJSY/e/g==
	caps mds = "allow *"
	caps mgr = "allow *"
	caps mon = "allow *"
	caps osd = "allow *"

4. Now debug (or secure shell) into a Worker node from step 1:

[OpenShift CLI]$ oc debug node/<Worker Node>
# chroot /host bash

5. Create a new directory to be used as the mount point for the CephFS volume:

[Worker Node]# mkdir /mnt/cephfs

6. Mount the CephFS directory using the Mon IP Addresses and the keyring we gathered from step 2:
   Note there is a slight difference in syntax when utilizing v1 versus v2 Non Addresses

Mon v1 syntax:

[Worker Node]# sudo mount -t ceph <mon_v1_ip_1>:6789,<mon_v1_ip_2>:6789,<mon_v1_ip_3>:6789:/ /mnt/cephfs -o name=admin,secret=<client-admin-key>

Mon v2 syntax:

[Worker Node]# sudo mount -t ceph <mon_v2_ip_1>:3300,<mon_v2_ip_2>:3300,<mon_v2_ip_3>:3300:/ /mnt/cephfs -o name=admin,secret=<client-admin-key>,ms_mode=prefer-secure

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.