Setting up TLS-enabled RGW in external Red Hat Ceph Storage for OpenShift Data Foundation

Updated

Configuring Object Store CR for TLS-enabled RGW

Prerequisites

Procedure

  1. Provide the TLS CA certificate and RGW endpoint details to the helper script, ceph-external-cluster-details-exporter.py.
    For more information, see Creating OpenShift Data Foundation cluster service for external storage.
# python3 ceph-external-clustergw-endpoint r-details-exporter.py  --rbd-data-pool-name <rbd block pool name> \
 --rgw-endpoint <ip_address>:<port> --rgw-tls-cert-path <file path containing cert>

This creates resource to create a Ceph Object Store CR such as Kubernetes secret containing the CA certificate. Any intermediate certificates need also to be stored in the certificate file.

  1. Create the Ceph Object Store CR using the following YAML:
apiVersion: ceph.rook.io/v1
kind: CephObjectStore
metadata:
  name: <name for cephobject store> #eg: external-store
  namespace: openshift-storage # namespace:cluster
spec:
  gateway:
    # The port on which **ALL** the gateway(s) are listening on.
    # Passing a single IP from a load-balancer is also valid.
    securePort: <port number> #eg: 442
    sslCertificateRef: ceph-rgw-tls-cert
    externalRgwEndpoints:
      - ip: 192.168.39.182
      # - hostname:

Note: This configuration can support multiple RGW endpoints in the CR. However, it is recommended to use only one RGW endpoint. If you want to use multiple RGW endpoints, use the load balancer.

  1. Create Ceph Object Store CRD:
# oc create -f <external-rgw-server.yaml>
  1. Confirm that the correct certificates are stored in the secret
# oc get secret ceph-rgw-tls-cert -n openshift-storage -o jsonpath='{.data.cert}' | base64 -d | openssl x509 -text

Accessing External RGW server in OpenShift Data Foundation

Accessing External RGW server using Object Bucket Claims

The S3 credentials such as AccessKey/Secret Key is stored in the secret generated by the Object Bucket Claim (OBC) creation and you can fetch the same by the following commands:

oc get secret <object bucket claim name> -o jsonpath='{.data.AWS_SECRET_ACCESS_KEY}' | base64 --decode)
oc get secret <object bucket claim name> -o jsonpath='{.data.AWS_ACCESS_KEY_ID}' | base64 --decode)

Similarly, you can fetch the endpoint details from the configmap of OBC:

# oc get cm <object bucket claim name> -o jsonpath='{.data.BUCKET_HOST}'
# oc get cm <object bucket claim name> -o jsonpath='{.data.BUCKET_PORT}'
# oc get cm <object bucket claim name> -o jsonpath='{.data.BUCKET_NAME}'

Accessing External RGW server using the Ceph Object Store User CR

You can fetch the S3 Credentials and endpoint details from the secret generated as part of the Ceph Object Store User CR:

get secret rook-ceph-object-user-<object-store-cr-name>-<object-user-cr-name> -o jsonpath='{.data.AccessKey}' | base64 --decode
get secret rook-ceph-object-user-<object-store-cr-name>-<object-user-cr-name> -o jsonpath='{.data.SecretKey}' | base64 --decode
get secret rook-ceph-object-user-<object-store-cr-name>-<object-user-cr-name> -o jsonpath='{.data.Endpoint}' | base64 --decode

Important: For both the access mechanisms, you can either request for new certificates from the administrator or reuse the certificates from the Kubernetes secret, ceph-rgw-tls-cert.

NOTE: This feature is dev-preview from ODF 4.13 - 4.15 and is Generally available from ODF 4.16 and higher versions.

SBR
Product(s)
Category
Tags
Article Type