Procedure

1. Edit the OpenStackControlPlane custom resource (CR) for editing:

   $ oc edit openstackcontrolplane openstack-control-plane

   Copy code to clipboard

2. Under the properties field under the Identity service (keystone) configuration, add the following:

     fernetMaxActiveKeys:
       default: <active_keys>
       description: FernetMaxActiveKeys - Maximum number of fernet token keys after rotation
       type: int
     fernetRotationDays:
       default: <days>

   Copy code to clipboard
   • Replace <active_keys> with the number of keys to keep active. The default is 5.
   • Replace <days> with the frequency with which to rotate your fernet keys.

Procedure

1. Create a PEM-formatted bundle, for example, mybundle.pem. Include all the CA certificates that you want OpenStack to trust.

2. Create a manifest file called cacerts.yaml that includes the mybundle.pem created in the previous step. Include all the certificates in chains of trust if applicable:

   apiVersion: v1
   kind: Secret
   metadata:
     name: cacerts
     namespace: openstack
   type: Opaque
   data:
     myBundleExample: <cat mybundle.pem | base64 -w0>
     CACertExample: <cat cacert.pem | base64 -w0>

   Copy code to clipboard
   • Replace mybundle.pem with the name of your certificate or certificate bundle. The results are pasted as the value of the myBundleExample field.
   • Replace cacert.pem with the name of your CA certificate.

3. Create the secret from the manifest file:

   $ oc apply -f cacerts.yaml

   Copy code to clipboard

4. Edit the openstack_control_plane.yaml custom resource (CR) file and add your bundle as the parameter for caBundleSecretName:

   apiVersion: core.openstack.org/v1beta1
   kind: OpenStackControlPlane
   metadata:
     name: myctlplane
   spec:
     tls:
       podLevel:
         enabled: true
       caBundleSecretName: cacerts

   Copy code to clipboard

5. Apply the control plane changes:

   $ oc apply -f openstack_control_plane.yaml

   Copy code to clipboard

6. Determine if TLS is enabled on each node set, by running the following command, which returns true if TLS is enabled on the specified node set:

   $ oc get openstackdataplanenodeset <node_set_name> -n <namespace> -o json | jq .items[0].spec.tlsEnabled

   Copy code to clipboard
   • Replace <node_set_name> with the name of the OpenStackDataPlaneNodeSet CR that the node belongs to.
   • Replace <namespace> with the namespace of the required Red Hat OpenStack Services on OpenShift (RHOSO) environment, for example, openstack.

7. If TLS is not enabled, you must enable it:

   1. Open the OpenStackDataPlaneNodeSet CR file for each node on the data plane, and enable TLS in each:

      apiVersion: dataplane.openstack.org/v1beta1
      kind: OpenStackDataPlaneNodeSet
      metadata:
        name: <node_set_name>
        namespace: openstack
      spec:
        tlsEnabled: true

      Copy code to clipboard

   2. Save the updated OpenStackDataPlaneNodeSet CR files and apply the updates:

      $ oc apply -f openstack_data_plane.yaml -n <namespace>

      Copy code to clipboard

8. Create a file on your workstation to define the OpenStackDataPlaneDeployment CR:

   apiVersion: dataplane.openstack.org/v1beta1
   kind: OpenStackDataPlaneDeployment
   metadata:
     name: <node_set_deployment_name>

   Copy code to clipboard
   • Replace <node_set_deployment_name> with the name of the OpenStackDataPlaneDeployment CR. This name must be unique, must consist of lower case alphanumeric characters, - (hyphen) or . (period), and must start and end with an alphanumeric character.

9. Add the OpenStackDataPlaneNodeSet CRs to the OpenStackDataPlaneDeployment CR file:

   spec:
     ...
     nodeSets:
       - <node_set_name>

   Copy code to clipboard
10. Save the OpenStackDataPlaneDeployment CR deployment file.

11. Deploy the modified OpenStackDataPlaneNodeSet CRs:

    $ oc create -f openstack_data_plane_deploy.yaml -n <namespace>

    Copy code to clipboard

    You can view the Ansible logs while the deployment executes:

    $ oc get pod -l app=openstackansibleee -w
    $ oc logs -l app=openstackansibleee -f --max-log-requests 10

    Copy code to clipboard

    If the oc logs command returns an error similar to the following error, increase the --max-log-requests value:

    error: you are attempting to follow 19 log streams, but maximum allowed concurrency is 10, use --max-log-requests to increase the limit

    Copy code to clipboard

12. Verify that the modified OpenStackDataPlaneNodeSet CRs are deployed:

    $ oc get openstackdataplanedeployment -n <namespace>
    NAME             	STATUS   MESSAGE
    openstack-data-plane   True     Setup Complete
    
    
    $ oc get openstackdataplanenodeset -n <namespace>
    NAME             	STATUS   MESSAGE
    openstack-data-plane   True     NodeSet Ready

    Copy code to clipboard

    For information about the meaning of the returned status, see Data plane conditions and states in the Deploying Red Hat OpenStack Services on OpenShift guide.

    If the status indicates that the data plane has not been deployed, then troubleshoot the deployment. For information, see Troubleshooting the data plane creation and deployment in the Deploying Red Hat OpenStack Services on OpenShift guide.

Procedure

1. Create a manifest file called cacerts.yaml that includes all CA certificates. Include all certificates in chains of trust if applicable:

   apiVersion: v1
   kind: Secret
   metadata:
     name: cacerts
     namespace: openstack
   type: Opaque
   data:
     myBundleExample: <cat mybundle.pem | base64 -w0>
     CACertExample: <cat cacert.pem | base64 -w0>

   Copy code to clipboard
   • Replace mybundle.pem with the name of your certificate or certificate bundle. The results are pasted as the value of the myBundleExample field.
   • Replace cacert.pem with the name of your CA certificate.

2. Create the secret from the manifest file:

   $ oc apply -f cacerts.yaml

   Copy code to clipboard

3. Create a manifest file for each secret named api_certificate_<service>_secret.yaml:

   apiVersion: v1
   kind: Secret
   metadata:
     name: api_certificate_<service>_secret
     namespace: openstack
   type: kubernetes.io/tls
   data:
     tls.crt: <cat tlscrt.pem | base64 -w0>
     tls.key: <cat tlskey.pem | base64 -w0>
     ca.crt: <cat cacrt.pem | base64 -w0>

   Copy code to clipboard
   • Replace <service> with the name of the service that this secret is for.
   • Replace tlscrt.pem with the name of your signed certificate.
   • Replace tlskey.pem with the name of your private key.
   • Replace cacrt.pem with the name of your CA certificate.

4. Create the secret

   $ oc apply -f api_certificate_<service>_secret.yaml

   Copy code to clipboard

5. Edit the openstack_control_plane.yaml custom resource and add your bundle as the parameter for caBundleSecretName:

   apiVersion: core.openstack.org/v1beta1
   kind: OpenStackControlPlane
   metadata:
     name: myctlplane
   spec:
     tls:
       podLevel:
         enabled: true
       caBundleSecretName: cacerts

   Copy code to clipboard

6. Apply the secret service certificates to each of the public services under the apiOverride field. For example enter the following for the Identity service (keystone):

   apiVersion: core.openstack.org/v1beta1
   kind: OpenStackControlPlane
   metadata:
     name: myctlplane
     namespace: openstack
   spec:
     ...
     keystone:
       apiOverride:
         tls:
           secretName: api_certificate_keystone_secret

   Copy code to clipboard

   The edits for the Compute service (nova) and noVNCProxy appear as the following:

   apiVersion: core.openstack.org/v1beta1
   kind: OpenStackControlPlane
   metadata:
     name: myctlplane
     namespace: openstack
   spec:
   ...
     nova:
       apiOverride:
         tls:
           secretName: api_certificate_nova_secret
         route: {}
       cellOverride:
         cell1:
           noVNCProxy:
             tls:
               secretName: api_certificate_novavncproxy_secret

   Copy code to clipboard

7. Apply the control plane changes

   $ oc apply -f openstack_control_plane.yaml

   Copy code to clipboard

Procedure

1. Create a signed certificate that includes the hostname for every service in the alt_names section:

   [alt_names]
   DNS.1 = barbican-public-openstack.apps.ocp.openstack.lab
   DNS.2 = cinder-public-openstack.apps.ocp.openstack.lab
   DNS.3 = glance-default-public-openstack.apps.ocp.openstack.lab
   DNS.4 = horizon-openstack.apps.ocp.openstack.lab
   DNS.5 = keystone-public-openstack.apps.ocp.openstack.lab
   DNS.6 = manila-public-openstack.apps.ocp.openstack.lab
   DNS.7 = neutron-public-openstack.apps.ocp.openstack.lab
   DNS.8 = nova-novncproxy-cell1-public-openstack.apps.ocp.openstack.lab
   DNS.9 = nova-public-openstack.apps.ocp.openstack.lab
   DNS.10 = placement-public-openstack.apps.ocp.openstack.lab

   Copy code to clipboard

2. Create a manifest file called cacerts.yaml that includes all CA certificates. Include all certificates in chains of trust if applicable:

   apiVersion: v1
   kind: Secret
   metadata:
     name: cacerts
     namespace: openstack
   type: Opaque
   data:
     myBundleExample: <cat mybundle.pem | base64 -w0>
     CACertExample: <cat cacert.pem | base64 -w0>

   Copy code to clipboard
   • Replace mybundle.pem with the name of your certificate or certificate bundle. The results are pasted as the value of the myBundleExample field.
   • Replace cacert.pem with the name of your CA certificate.

3. Create the secret from the manifest file:

   $ oc apply -f cacerts.yaml

   Copy code to clipboard

4. Create a manifest file for a secret named certificate-secret.yaml:

   apiVersion: v1
   kind: Secret
   metadata:
     name: certificate-secret
     namespace: openstack
   type: kubernetes.io/tls
   data:
     tls.crt: <cat tlscrt.pem | base64 -w0>
     tls.key: <cat tlskey.pem | base64 -w0>
     ca.crt: <cat cacrt.pem | base64 -w0>

   Copy code to clipboard
   • Replace tlscrt.pem with the name of your signed certificate.
   • Replace tlskey.pem with the name of your private key.
   • Replace cacrt.pem with the name of your CA certificate.

5. Create the secret

   $ oc apply -f certificate-secret.yaml

   Copy code to clipboard

6. Edit the openstack_control_plane.yaml custom resource and add your bundle as the parameter for caBundleSecretName:

   apiVersion: core.openstack.org/v1beta1
   kind: OpenStackControlPlane
   metadata:
     name: myctlplane
   spec:
     tls:
       podLevel:
         enabled: true
       caBundleSecretName: cacerts

   Copy code to clipboard

7. Apply the secret service certificates to each of the public services under the apiOverride field. For example, enter the following for the Identity service (keystone):

   apiVersion: core.openstack.org/v1beta1
   kind: OpenStackControlPlane
   metadata:
     name: myctlplane
     namespace: openstack
   spec:
     ...
     keystone:
       apiOverride:
         tls:
           secretName: certificate-secret

   Copy code to clipboard

   The edits for the Compute service (nova) and NoVNCProxy appear as the following:

   apiVersion: core.openstack.org/v1beta1
   kind: OpenStackControlPlane
   metadata:
     name: myctlplane
     namespace: openstack
   spec:
   ...
     nova:
       apiOverride:
         tls:
           secretName: certificate-secret
         route: {}
       cellOverride:
         cell1:
           NoVNCProxy:
             tls:
               secretName: certificate-secret

   Copy code to clipboard

8. Apply the control plane changes

   $ oc apply -f openstack_control_plane.yaml

   Copy code to clipboard

Procedure

1. Log onto OpenShift with global administrative permissions.

2. Extract the ca cert for the public endpoints from the rootca-public secret.

   $ oc get secret rootca-public -o json | jq -r '.data."ca.crt"' | base64 -d > ca.crt

   Copy code to clipboard
3. Transfer the ca.crt file to the client that accesses the OpenStack API.

4. Update your authentication file with the path to ca.crt.

   1. If you use a clouds.yml authentication file, add the cacert parameter:

      clouds:
          secure:
              cacert: </path/to/ca.crt>

      Copy code to clipboard
      • Replace </path/to/ca.crt> with the absolute path and name of the CA cert on your system.

   2. If you use a resource credentials file, update the file with the exported CACERT variable:

      $ export OC_CACERT=</path/to/ca.crt>

      Copy code to clipboard
      • Replace </path/to/ca.crt> with the absolute path and name of the CA cert on your system.

Procedure

1. Create a custom issuer in a file named rootca-custom.yaml:

   apiVersion: cert-manager.io/v1
   kind: Issuer
   metadata:
     name: <issuer_name>
   spec:
     ca:
       secretName: <secret_name>

   Copy code to clipboard
   • Replace <issuer_name> with the name of your custom issuer, for example, rootca-ingress-custom.
   • Replace <secret_name> with the name of the Secret CR used by the certificate for your custom issuer. If you do not include a secret, one is created automatically.

2. Create a certificate in a file named ca-issuer-certificate.yaml:

   apiVersion: cert-manager.io/v1
   kind: Certificate
   metadata:
     name: <issuer_name>
   spec:
     commonName: <issuer_name>
     isCA: true
     duration: <hours>
     privateKey:
       algorithm: RSA
       size: 3072
     issuerRef:
       name: selfsigned-issuer
       kind: Issuer
     secretName: <secret-name>

   Copy code to clipboard
   • Replace <issuer_name> with the name of your custom issuer. This matches the issuer created in the first step.
   • Replace <hours> with the duration in hours, for example, a value of 87600h is equivalent to 3650 days, or about 10 years.
   • Replace <secret_name> with the name of the Secret CR used by the certificate for your custom issuer. If you do not include a secret, one is created automatically.

3. Create the issuer and certificate:

   $ oc create -f rootca-custom.yaml
   $ oc create -f ca-issuer-certificate.yaml

   Copy code to clipboard

4. Add the custom issuer to the TLS service definition in the control plane CR file.

   1. If your custom issuer is an ingress issuer, the customer issuer is defined under the ingress attribute as shown below:

      apiVersion: core.openstack.org/v1beta1
      kind: OpenStackControlPlane
      metadata:
        name: openstack-control-plane
      spec:
        tls:
           ingress:
             enabled: true
             ca:
               customIssuer: <issuer_name>
         ...

      Copy code to clipboard
      • Replace <issuer_name> with the name of your custom issuer. This matches the issuer created in the first step.

   2. If your custom issuer is an internal issuer, the custom issuer is defined at the pod level under the internal attribute as shown below:

      apiVersion: core.openstack.org/v1beta1
      kind: OpenStackControlPlane
      metadata:
        name: myctlplane
      spec:
        tls:
           ingress:
             enabled: true
           podLevel:
             enabled: true
             internal:
               ca:
                 customIssuer: <issuer_name>

      Copy code to clipboard
      • Replace <issuer_name> with the name of your custom issuer. This matches the issuer created in the first step.

Procedure

1. Open your OpenStackControlPlane custom resource (CR) file, openstack_control_plane.yaml, on your workstation.

2. Add the following spec.tls configuration, if not already present:

   spec:
     tls:
       ingress:
         ca:
           duration: 87600h0m0s
         cert:
           duration: 43800h0m0s
         enabled: true
       podLevel:
         enabled: true
         internal:
           ca:
             duration: 87600h0m0s
           cert:
             duration: 43800h0m0s
         libvirt:
           ca:
             duration: 87600h0m0s
           cert:
             duration: 43800h0m0s
         ovn:
           ca:
             duration: 87600h0m0s
           cert:
             duration: 43800h0m0s

   Copy code to clipboard
   • If the tls configuration is already present in the CR file, then ensure that the podLevel is enabled.

3. Update the control plane:

   $ oc apply -f openstack_control_plane.yaml -n openstack

   Copy code to clipboard

4. The rabbitmq pods cannot change the TLS configuration on an operating environment, therefore you must delete the existing rabbitmq pods to update the control plane with the new rabbitmq pods that have TLS enabled:

   $ oc delete pod -n openstack -l app.kubernetes.io/component=rabbitmq

   Copy code to clipboard

5. Wait for the control plane to be ready:

   $ oc wait openstackcontrolplane -n openstack --for=condition=Ready --timeout=400s -l core.openstack.org/openstackcontrolplane

   Copy code to clipboard

   While waiting for the control plane to be ready, new workloads cannot be created and existing workloads cannot be managed with the OpenStack API. The nova-compute service on the data plane nodes cannot connect to the cell1 rabbitmq and reports as down:

   $ oc rsh openstackclient
   $ openstack compute service list -c Binary -c Host -c Status -c State
   +----------------+-------------------------------------+---------+-------+
   | Binary         | Host                                | Status  | State |
   +----------------+-------------------------------------+---------+-------+
   | nova-conductor | nova-cell0-conductor-0              | enabled | up    |
   | nova-scheduler | nova-scheduler-0                    | enabled | up    |
   | nova-conductor | nova-cell1-conductor-0              | enabled | up    |
   | nova-compute   | edpm-compute-0.ctlplane.example.com | enabled | down  |
   | nova-compute   | edpm-compute-1.ctlplane.example.com | enabled | down  |
   +----------------+-------------------------------------+---------+-------+

   Copy code to clipboard

   The OVN controller and the OVN metadata agent cannot connect to the southbound database:

   $ openstack network agent list -c 'Agent Type' -c Host -c Alive -c State
   +------------------------------+-------------------------------------+-------+-------+
   | Agent Type                   | Host                                | Alive | State |
   +------------------------------+-------------------------------------+-------+-------+
   | OVN Controller Gateway agent | crc                                 | :-)   | UP    |
   | OVN Controller agent         | edpm-compute-1.ctlplane.example.com | XXX   | UP    |
   | OVN Metadata agent           | edpm-compute-1.ctlplane.example.com | XXX   | UP    |
   | OVN Controller agent         | edpm-compute-0.ctlplane.example.com | XXX   | UP    |
   | OVN Metadata agent           | edpm-compute-0.ctlplane.example.com | XXX   | UP    |
   +------------------------------+-------------------------------------+-------+-------+

   Copy code to clipboard
   Note

   The existing workload is not impacted if workload traffic is not routed through the control plane.

6. Open the OpenStackDataPlaneNodeSet CR file for each node on the data plane, and enable TLS in each:

   apiVersion: dataplane.openstack.org/v1beta1
   kind: OpenStackDataPlaneNodeSet
   metadata:
     name: <node_set_name>
     namespace: openstack
   spec:
     tlsEnabled: true

   Copy code to clipboard
   • Replace <node_set_name> with the name of the OpenStackDataPlaneNodeSet CR that the node belongs to.

7. Save the updated OpenStackDataPlaneNodeSet CR files and apply the updates:

   $ oc apply -f openstack_data_plane.yaml -n openstack

   Copy code to clipboard

8. Check that TLS is enabled on each node set:

   $ oc get openstackdataplanenodeset <node_set_name> -n openstack -o json | jq .items[0].spec.tlsEnabled
   true

   Copy code to clipboard

9. Create a file on your workstation to define the OpenStackDataPlaneDeployment CR:

   apiVersion: dataplane.openstack.org/v1beta1
   kind: OpenStackDataPlaneDeployment
   metadata:
     name: <node_set_deployment_name>

   Copy code to clipboard
   • Replace <node_set_deployment_name> with the name of the OpenStackDataPlaneDeployment CR. The name must be unique, must consist of lower case alphanumeric characters, - (hyphen) or . (period), and must start and end with an alphanumeric character.
   Tip

   Give the OpenStackDataPlaneDeployment CR file a descriptive name that indicates the purpose of the modified node set.

10. Add the OpenStackDataPlaneNodeSet CRs that you modified to enable TLS:

    spec:
      nodeSets:
        - <node_set_name>

    Copy code to clipboard
    • Provide the required <node_set_name> for each node on the data plane.
11. Save the OpenStackDataPlaneDeployment CR deployment file.

12. Deploy the modified OpenStackDataPlaneNodeSet CRs:

    $ oc create -f openstack_data_plane_deploy.yaml -n openstack

    Copy code to clipboard

    You can view the Ansible logs while the deployment executes:

    $ oc get pod -l app=openstackansibleee -n openstack -w
    $ oc logs -l app=openstackansibleee -f --max-log-requests 10 -n openstack

    Copy code to clipboard

    If the oc logs command returns an error similar to the following error, increase the --max-log-requests value:

    error: you are attempting to follow 19 log streams, but maximum allowed concurrency is 10, use --max-log-requests to increase the limit

    Copy code to clipboard

13. Verify that the modified OpenStackDataPlaneNodeSet CRs are deployed:

    $ oc get openstackdataplanedeployment -n openstack
    NAME             	STATUS   MESSAGE
    openstack-data-plane   True     Setup Complete
    
    
    $ oc get openstackdataplanenodeset -n openstack
    NAME             	STATUS   MESSAGE
    openstack-data-plane   True     NodeSet Ready

    Copy code to clipboard

    For information about the returned status, see Data plane conditions and states in the Deploying Red Hat OpenStack Services on OpenShift guide.

    If the status indicates that the data plane has not been deployed, then troubleshoot the deployment. For information, see Troubleshooting the data plane creation and deployment in the Deploying Red Hat OpenStack Services on OpenShift guide.

14. Verify that the nova-compute service is connected again to TLS rabbitmq:

    $ oc rsh openstackclient
    $ openstack compute service list -c Binary -c Host -c Status -c State
    +----------------+-------------------------------------+---------+-------+
    | Binary         | Host                                | Status  | State |
    +----------------+-------------------------------------+---------+-------+
    | nova-conductor | nova-cell0-conductor-0              | enabled | up    |
    | nova-scheduler | nova-scheduler-0                    | enabled | up    |
    | nova-conductor | nova-cell1-conductor-0              | enabled | up    |
    | nova-compute   | edpm-compute-0.ctlplane.example.com | enabled | up    |
    | nova-compute   | edpm-compute-1.ctlplane.example.com | enabled | up    |
    +----------------+-------------------------------------+---------+-------+

    Copy code to clipboard

15. Verify that the OVN agents are running again:

    $ openstack network agent list -c 'Agent Type' -c Host -c Alive -c State
    +------------------------------+-------------------------------------+-------+-------+
    | Agent Type                   | Host                                | Alive | State |
    +------------------------------+-------------------------------------+-------+-------+
    | OVN Controller Gateway agent | crc                                 | :-)   | UP    |
    | OVN Controller agent         | edpm-compute-1.ctlplane.example.com | :-)   | UP    |
    | OVN Metadata agent           | edpm-compute-1.ctlplane.example.com | :-)   | UP    |
    | OVN Controller agent         | edpm-compute-0.ctlplane.example.com | :-)   | UP    |
    | OVN Metadata agent           | edpm-compute-0.ctlplane.example.com | :-)   | UP    |
    +------------------------------+-------------------------------------+-------+-------+

    Copy code to clipboard

Procedure

1. Open your OpenStackControlPlane custom resource (CR) file, openstack_control_plane.yaml, on your workstation.

2. Add the following spec.tls configuration, if not already present:

   spec:
     tls:
       ingress:
         enabled: false
       podLevel:
         enabled: false

   Copy code to clipboard

3. Update the control plane:

   $ oc apply -f openstack_control_plane.yaml

   Copy code to clipboard

4. Open the OpenStackDataPlaneNodeSet CR file for each node on the data plane, and disable TLS by setting spec.tlsEnabled to false:

   apiVersion: dataplane.openstack.org/v1beta1
   kind: OpenStackDataPlaneNodeSet
   metadata:
     name: <node_set_name>
     namespace: openstack
   spec:
     tlsEnabled: false

   Copy code to clipboard
   • Replace <node_set_name> with the name of the OpenStackDataPlaneNodeSet CR that the node belongs to.

5. Save the updated OpenStackDataPlaneNodeSet CR files and apply the updates:

   $ oc apply -f openstack_data_plane.yaml

   Copy code to clipboard

6. Verify that TLS is disabled on every node set:

   $ oc get openstackdataplanenodeset <node_set_name> -n openstack -o json | jq .items[0].spec.tlsEnabled

   Copy code to clipboard

Procedure

1. Create an OpenStack domain:

   $ openstack domain create <name>

   Copy code to clipboard
   • Replace <name> with the name of your OpenStack domain.

2. Create a keystone-domains secret called keystone-domains.yaml. This secret is mounted into the /etc/keystone/domains configuration directory:

   apiVersion: v1
   kind: Secret
   metadata:
     name: keystone-domains
     namespace: openstack
   type: Opaque
   stringData:
       keystone.<domain_name>.conf: |
           [identity]
           driver = ldap
           [ldap]
           url = ldaps://localhost
           user = =openstack,ou=Users,dc=director,dc=example,dc=com
           password = RedactedComplexPassword
           suffix = dc=domain,dc=example,dc=com
           user_tree_dn = ou=Users,dc=domain,dc=example,dc=com
           user_objectclass = person
           group_tree_dn = ou=Groups,dc=example,dc=org
           group_objectclass = groupOfNames
           use_tls = True

   Copy code to clipboard

3. Create the secret:

   $ oc apply -f keystone-domain-name.yaml

   Copy code to clipboard

4. Open your OpenStackCustomResource custom resource (CR) file and add the secret by using the extraMounts field:

   apiVersion: core.openstack.org/v1beta1
   kind: OpenStackControlPlane
   metadata:
     name: openstack
   spec:
     keystone:
       template:
         customServiceConfig: |
           [identity]
           domain_specific_drivers_enabled = True
         extraMounts:
         - name: v1
           region: r1
             extraVol:
               - propagation:
                 - Keystone
                 extraVolType: Conf
                 volumes:
                 - name: keystone-domains
                   secret:
                     secretName: keystone-domains
                 mounts:
                 - name: keystone-domains
                   mountPath: "/etc/keystone/domains"
                   readOnly: true

   Copy code to clipboard

5. Apply the changes to your OpenStack control plane CR:

   $ oc apply -f openstack_control_plane.yaml

   Copy code to clipboard

Procedure

1. Use DNF to install ansible-role-rhoso-luna-hsm:

   $ sudo dnf -y install ansible-role-rhoso-luna-hsm

   Copy code to clipboard
2. Place the Luna minimal client image for Linux in a known location. In this procedure, the image is placed in /opt/luna.

3. Move a copy of the Luna minimal client for Linux tarball to /opt/luna:

   $ mv <LunaClient-Minimal-10.7.2.x86_64.tar> /opt/luna

   Copy code to clipboard
   • Replace <LunaClient-Minimal-10.7.2.x86_64.tar> with the name of your Luna Minimal client for Linux tarball.

4. Create a playbook called custom-image.yaml that creates the custom Key Manager image:

   ---
   - name: Create and upload the custom Key Manager image
     ansible.builtin.include_role:
       name: rhoso_luna_hsm
       tasks_from: create_image
     vars:
       barbican_src_image_registry: "quay.io:5001"
       barbican_src_image_namespace: "openstack-k8s-operators"
       barbican_src_image_tag: "latest"
       barbican_dest_image_registry: "<my_registry_url>:5001"
       barbican_dest_image_namespace: "openstack-k8s-operators"
       barbican_dest_image_tag: "luna-custom"
       image_registry_verify_tls: "<true|false>"
       luna_minclient_src: "file:///opt/luna/<filename>"
   ---

   Copy code to clipboard
   • Replace <my_registry_url> with the URL for your registry.
   • Replace <true|false> with either true or false based on the requirements of your image registry.
   • Replace <filename> with the name of your source image, for example: LunaClient-Minimal-10.7.2.x86_64.tar.

5. Run the playbook:

   $ ansible-playbook custom-image.yaml

   Copy code to clipboard

Procedure

1. Place the Luna certificate and key into the /opt/luna directory tree:

   $ cp <luna_client_name>.pem /opt/luna
   $ cp <luna_client_name>Key.pem /opt/luna

   Copy code to clipboard
   • Replace <luna_client_name> with the name of your Luna certificate.

2. Download the server certificate from the HSM device:

   $ scp -O <hsm-device.examle.com:server.pem> /opt/luna/

   Copy code to clipboard

3. Optional: If you have more than one HSM for HA, get every cert from the HSM and concatenate them into a single file:

   $ scp -O <hsm-device-01.examle.com:server-01.pem> /opt/luna/
   $ scp -O <hsm-device-02.examle.com:server-02.pem> /opt/luna/
   $ cat /opt/luna/cert/server-01.pem > /opt/luna/CAFile.pem
   $ cat /opt/luna/cert/server-02.pem >> /opt/luna/CAFile.pem

   Copy code to clipboard

4. Update your Crystoki.conf file to look similar to the following:

   Note

   The contents from the LunaClient-Minimal tarball is extracted to the /usr/local/luna/ directory in the Key Manager container. You must update the paths in your Crystoki.conf file to match this example.

   Chrystoki2 = {
     LibUNIX = /usr/local/luna/libs/64/libCryptoki2.so;
     LibUNIX64 = /usr/local/luna/libs/64/libCryptoki2.so;
   }
   
   Luna = {
     DefaultTimeOut = 500000;
     PEDTimeout1 = 100000;
     PEDTimeout2 = 200000;
     PEDTimeout3 = 10000;
     KeypairGenTimeOut = 2700000;
     CloningCommandTimeOut = 300000;
     CommandTimeOutPedSet = 720000;
   }
   
   CardReader = {
     RemoteCommand = 1;
   }
   
   Misc = {
     PE1746Enabled = 0;
     ToolsDir = ./bin/64;
     PartitionPolicyTemplatePath = ./ppt/partition_policy_templates;
     ProtectedAuthenticationPathFlagStatus = 0;
     MutexFolder = ./lock;
   }
   
   LunaSA Client = {
     ReceiveTimeout = 20000;
     SSLConfigFile = /usr/local/luna/openssl.cnf;
     ClientPrivKeyFile = /usr/local/luna/<luna_client_name>Key.pem;
     ClientCertFile = /usr/local/luna/<luna_client_name>.pem;
     ServerCAFile = /usr/local/luna/CAFile.pem;
     NetClient = 1;
     TCPKeepAlive = 1;
     ServerName00 = <ip_address>;
     ServerPort00 = 1792;
     ServerHtl00 = 0;
   }

   Copy code to clipboard
   • Replace <luna_client_name> with the name of your Luna certificate.
   • Replace <ip_address> with the IP address of your Luna HSM.

5. Optional: If you are configuring HA, you must include additional entries for the IP addresses of each HSM, as well as configurations for the VirtualToken, HASynchronize, and HAConfigurations parameters:

   ...
     ServerName00 = <ip_address>;
     ServerPort00 = 1792;
     ServerHtl00 = 0;
     ServerName01 = <ip_address>;
     ServerPort01 = 1792;
     ServerHtl01 = 0;
   }
   
   VirtualToken = {
     VirtualToken00Label = myHAGroup;
     VirtualToken00SN = <virtual_token_sn>;
     VirtualToken00Members = <virtual_token_member>,<virtual_token_member>;
   }
   
   HASynchronize = {
     myHAGroup = 1;
   }
   
   HAConfiguration = {
     haLogStatus = enabled;
   }

   Copy code to clipboard
   • Replace <virtual_token_sn> with the serial number of your first partition prepended by a 1. For example, for partition 545000014, use a value of 1545000014
   • Replace <virtual_token_member> with the serial numbers of the partitions from the HSMs you are using.

6. Move the chrystoki.conf configuration file to /opt/luna:

   $ mv chrystoki.conf /opt/luna

   Copy code to clipboard

7. Create an Ansible playbook called create-luna-secrets.yaml to create the required secrets:

   ---
   - name: Create secrets with the HSM certs and hsm-login credentials
   ansible.builtin.include_role:
       name: rhoso_luna_hsm
       tasks_from: create_secrets
   vars:
       luna_client_name: <luna_client_name>
       chrystoki_conf_src: "/opt/luna/chrystoki.conf"
       luna_server_cert_src: "/opt/luna/<server.pem>"
       luna_client_cert_src: "/opt/luna/"
       luna_partition_password: "<my_partion_password>"
       kubeconfig_path: "<kubeconfig_path>"
       oc_dir: "<path_to_oc>"
       luna_data_secret: "luna_data_secret"
       login_secret: "login_secret"
   ---

   Copy code to clipboard
   • Replace <luna_client_name> with the name of your Luna certificate.
   • Replace <server.pem> with the name of your server certificate.
   • Replace <my_partion_password> with your HSM partition password.
   • Replace <kubeconfig_path> with the path to your .kube configuration file. For example: $HOME/.kube/config.
   • Replace <path_to_oc> with the output of which oc.

8. Run the Ansible play book:

   $ ansible-playbook create-luna-secrets.yaml

   Copy code to clipboard

Procedure

1. Create a CR file with the following contents:

   apiVersion: core.openstack.org/v1beta1
   kind: OpenStackVersion
   metadata:
     name: openstack-galera-network-isolation
     namespace: openstack
   spec:
     customContainerImages:
         barbicanAPIImage: <api_image>
         barbicanWorkerImage: <worker_image>

   Copy code to clipboard
   • Replace <api_image> with the registry and path to the custom barbicanAPIImage.
   • Replace <worker_image> with the registry and path to the custom barbicanWorkerImage.

2. Apply the OpenStackVersion CR:

   $ oc apply -f <filename>

   Copy code to clipboard
   • Replace <filename> with the OpenStackVersion CR file name.

Procedure

1. Configure the OpenStackControlPlane CR:

   1. Optional: If you have saved secrets that are using RHOSO Key Manager simple_crypto, keep those secrets available by enabling multiple back ends:

      spec:
        barbican:
          apiOverride:
            route: {}
          enabled: true
          template:
            globalDefaultSecretStore: pkcs11
            enabledSecretStores:
              - pkcs11
              - simple_crypto

      Copy code to clipboard

   2. Configure the Key Manager service for use with the Luna HSM:

      spec:
        barbican:
          apiOverride:
            route: {}
          enabled: true
          template:
            globalDefaultSecretStore: pkcs11
            enabledSecretStores:
              - pkcs11
              - simple_crypto
            apiTimeout: 90
            barbicanAPI:
              apiTimeout: 0
              customServiceConfig: |
                [secretstore:pkcs11]
                secret_store_plugin = store_crypto
                crypto_plugin = p11_crypto
                [p11_crypto_plugin]
                plugin_name = PKCS11
                library_path = /usr/local/luna/lib/libCryptoki2.so
                token_serial_number = <serial_number>
                mkek_label = <mkek_label>
                hmac_label = <hmac_label>
                encryption_mechanism = CKM_AES_GCM
                aes_gcm_generate_iv = true
                hmac_key_type = CKK_GENERIC_SECRET
                hmac_keygen_mechanism = CKM_GENERIC_SECRET_KEY_GEN
                hmac_keywrap_mechanism = CKM_AES_KEY_WRAP_KWP
                key_wrap_mechanism = true
                key_wrap_generate_iv = true
                always_set_cka_sensitive = true
                os_locking_ok = false
              pkcs11:
                loginSecret: "login_secret"
                clientDataSecret: "luna_data_secret"
                clientDataPath: /usr/local/luna/config

      Copy code to clipboard
      • Replace <serial_number> with the token serial number of your HSM. If you are using HA, you must replace <serial_number> with the virtual token serial number. For more information, see Creating secrets for the Key Manager service.
      • Replace <mkek_label> with a user-defined label. If you have already defined this label, you must use the same one.

      • Replace <hmac_label> with a user-defined label. If you have already defined this label, you must use the same one.

        Note

        Use one of the following options to identify the HSM that you can use. These options are mutually exclusive and have the following order of precedence:

        Parameter	Value	Precedence
        token_serial_number	<serial_number>	1 - Highest
        token_labels	Comma delimited lists	2 - Middle
        slot-id	<slot_id>	3 - Lowest

2. Deploy the OpenStackControlPlane CR:

   $ oc apply -f openstack_control_plane.yaml

   Copy code to clipboard

Procedure

1. Use DNF to install ansible-role-rhoso-proteccio-hsm:

   $ sudo dnf -y install ansible-role-rhoso-proteccio-hsm

   Copy code to clipboard

2. Place the Trustway client image for Linux, as well as the client cert and the client key into the /opt/proteccio directory tree.

   $ cp <trustway_client_cert>.crt /opt/proteccio
   $ cp <Trustway_client_key>.key /opt/proteccio
   $ cp  <Proteccio3.06.05.iso> /opt/proteccio

   Copy code to clipboard
   • Replace <trustway_client_cert> with the file name of your client certificate.
   • Replace <trustway_client_key> with the file name of your client key.
   • Replace <Proteccio3.06.05.iso> with the name of your Trustway client for Linux ISO.
3. Retrieve the server certificate from the HSM device, and copy it to the /opt/proceccio directory. For more information on retrieving the server certificate from your Proteccio HSM, see the vendor documentation.
4. Optional: If you have more than one HSM for HA, get every certificate for each of the HSMs and put them altogether in the /opt/proteccio directory.

5. Update your proteccio.rc file to look similar to the following:

   [PROTECCIO]
   IPaddr=<Trustway_HSM_IP_address>
   SSL=1
   SrvCert=<HSM_Certificate_Name>.CRT
   
   [CLIENT]
   Mode=0
   LoggingLevel=7
   LogFile=/var/log/barbican/proteccio.log
   StatusFile=/var/log/barbican/HSM_Status.log
   ClntKey=<Client_Certificate_Name>.key
   ClntCert=<Client_Certificate_Name>.crt

   Copy code to clipboard
   • Replace <Trustway_HSM_IP_Address> with the IP address of your Trustway HSM.
   • Replace <HSM_Certificate_Name> with the name of your Trustway certificate.
   • In the file above, Mode=0 means that only a single HSM device is in place.
   • Replace <Client_Certificate_Name> with your client certificate name.

6. Optional: If you are configuring HA, you must include additional entries for the IP addresses of each HSM. Each new HSM must be inside of a [PROTECCIO] section. Additionally, you much change the Mode parameter inside the [CLIENT] to a value of either 1 or 2. For more information, see the official Eviden documentation.

   [PROTECCIO]
   IPaddr=<Trustway_HSM-2_IP_address>
   SSL=1
   SrvCert=<HSM-2_Certificate_Name>.CRT
   
   [CLIENT]
   Mode=2

   Copy code to clipboard
   • Replace <Trustway_HSM-2_IP_Address> with the IP address of your second Trustway HSM.
   • Create a new [PROTECCIO] section with the corresponding parameters for every subsequent Trustway unit you have in your environment.

7. Move the proteccio.rc configuration file to /opt/proteccio:

   $ mv proteccio.rc /opt/proteccio

   Copy code to clipboard

8. Create a playbook called ansible-proteccio.yaml with the following contents:

     vars:
       Trustway_client_name: <name>
       Trustway_server_cert_src: "/opt/trustway/<server.pem>"
       Trustway_partition_password: "<password>"
       Trustway_data_secret: "Trustway_data_secret"
       login_secret: "login_secret"
       barbican_dest_image_namespace: "<namespace>"
       proteccio_client_src: "file:///opt/proteccio/<iso_file>"
       proteccio_password: "{{ PIN to log into proteccio }}"
       kubeconfig_path: "<kubeconfig_path"
       oc_dir: "<directory>"
     roles:
       - rhoso_proteccio_hsm

   Copy code to clipboard
   • Replace <name> with the name of your Trustway certificate.
   • Replace <server.pem> with the name of your server certificate.
   • Replace <password> with your HSM partition password.
   • Replace <namespace> with your account name for Quay.io or other container registry.
   • Replace the contents of <iso_file> with the name of the Proteccio client ISO file.
   • Replace the contents of <kubeconfig_path> with the full path to your OpenShift’s configuration file.
   • Replace <directory> with the full path to the OpenShift Client location.

9. Run the playbook:

   $ ansible-playbook ansible-proteccio.yaml

   Copy code to clipboard

Procedure

1. Create a CR file with the following contents:

   apiVersion: core.openstack.org/v1beta1
   kind: OpenStackVersion
   metadata:
     name: openstack-galera-network-isolation
     namespace: openstack
   spec:
     customContainerImages:
         barbicanAPIImage: <api_image>
         barbicanWorkerImage: <worker_image>

   Copy code to clipboard
   • Replace <api_image> with the registry and path to the custom barbicanAPIImage.
   • Replace <worker_image> with the registry and path to the custom barbicanWorkerImage.

2. Apply the OpenStackVersion CR:

   $ oc apply -f <filename>

   Copy code to clipboard
   • Replace <filename> with the OpenStackVersion CR file name.

Procedure

1. Configure the Key Manager service within your OpenStackControlPlane CR for use with the Trustway HSM:

   spec:
     barbican:
       apiOverride:
         route: {}
       enabled: true
       template:
         globalDefaultSecretStore: pkcs11
         enabledSecretStores:
           - pkcs11
           - simple_crypto
         apiTimeout: 90
         barbicanAPI:
           apiTimeout: 0
           customServiceConfig: |
             [secretstore:pkcs11]
             secret_store_plugin = store_crypto
             crypto_plugin = p11_crypto
             [p11_crypto_plugin]
             plugin_name = PKCS11
             library_path = /opt/tw_proteccio/lib/libnethsm.so
             token_labels = <token_label>
             mkek_label = <mkek_label>
             hmac_label = <hmac_label>
             encryption_mechanism = CKM_AES_CBC
             hmac_key_type = CKK_GENERIC_SECRET
             hmac_keygen_mechanism = CKM_GENERIC_SECRET_KEY_GEN
             hmac_mechanism = CKM_SHA256_HMAC
             key_wrap_mechanism = CKM_AES_CBC_PAD
             key_wrap_generate_iv = true
             always_set_cka_sensitive = true
             os_locking_ok = false
   
           pkcs11:
             loginSecret: "login_secret"
             clientDataSecret: "proteccio-data"
             clientDataPath: /etc/proteccio

   Copy code to clipboard
   • Replace <token_label> with the token label of your HSM. If you are using HA, you must replace <token_label> with the virtual token serial number.
   • Replace <mkek_label> with a user-defined label. If you have already defined this label, you must use the same one.

   • Replace <hmac_label> with a user-defined label. If you have already set this up, you must use the same label.

     Note

     Use one of the following options to identify the HSM that you can use. These options are mutually exclusive and have the following order of precedence:

     Parameter	Value	Precedence
     token_serial_number	<serial_number>	1 - Highest
     token_labels	Comma delimited lists	2 - Middle
     slot-id	<slot_id>	3 - Lowest

2. Optional: If you have saved secrets that are using RHOSO Key Manager simple_crypto, keep those secrets available by enabling multiple back ends:

   spec:
     barbican:
       apiOverride:
         route: {}
       enabled: true
       template:
         globalDefaultSecretStore: pkcs11
         enabledSecretStores:
           - pkcs11
           - simple_crypto

   Copy code to clipboard

3. Deploy the OpenStackControlPlane CR:

   $ oc apply -f openstack_control_plane.yaml

   Copy code to clipboard

Procedure

1. Retrieve the Identity service (keystone) endpoint:

   $ oc get keystoneapis.keystone.openstack.org -o json | jq '.items[0].status.apiEndpoints.public'

   Copy code to clipboard

2. Provide your SSO administrator with the following redirect URIs as well as the web origin.

   https://<keystoneURL>/v3/auth/OS-FEDERATION/identity_providers/<idp_name>/protocols/openid/websso/
   https://<keystoneURL>/v3/auth/OS-FEDERATION/websso/openid
   
   webOrigins: https://<keystoneURL>

   Copy code to clipboard
   • Replace <keystoneURL> with the URL retrieved in step 1. This url must end in a trailing /.

   • Replace <idp_name> with a value of your choosing, for example, kcipaIDP.

     In response, your SSO administrator provides you with a ClientID and a ClientSecret.

     Note

     The chosen <idp_name> value must match all referenced <idp_name> values in this procedure.

3. Retrieve the Memcached hostname:

   1. For an IPv4 deployment run the following command:

      $ oc get memcacheds.memcached.openstack.org -n openstack -o json | jq -r '.items[0].status.serverList[0] | split(":")[0]'

      Copy code to clipboard

   2. For an IPv6 deployment, run the following command:

      $ oc get memcacheds.memcached.openstack.org -n openstack -o json | jq -r '.items[0].status.serverListWithInet[0]'

      Copy code to clipboard

4. Create a keystone-httpd-override.yaml CR file and add the following configuration:

   apiVersion: v1
   kind: Secret
   metadata:
     name: keystone-httpd-override
     namespace: openstack
   type: Opaque
   stringData:
     federation.conf: |
       # Example OIDC directives for the *public* endpoint
       OIDCClaimPrefix "OIDC-"
       OIDCScope "openid email profile"
       OIDCClaimDelimiter ";"
       OIDCPassUserInfoAs "claims"
       OIDCPassClaimsAs "both"
       OIDCClientID "<my_client_id>"
       OIDCClientSecret "<my_client_secret>"
       OIDCCryptoPassphrase "<crypto_pass>"
       OIDCProviderMetadataURL <metadata_url>
       OIDCResponseType "id_token"
       OIDCOAuthClientID "my_oauth_client_id"
       OIDCOAuthClientSecret "12345678"
       OIDCOAuthIntrospectionEndpoint "<https://my_oauth_introspection_endpoint>"
       OIDCRedirectURI "{{ .KeystoneEndpointPublic }}/v3/auth/OS-FEDERATION/identity_providers/<idp_name>/protocols/openid/websso/"
   
       <LocationMatch "/v3/auth/OS-FEDERATION/identity_providers/<idp_name>/protocols/openid/websso">
           AuthType "openid-connect"
           Require valid-user
       </LocationMatch>
   
       <Location ~ "/v3/OS-FEDERATION/identity_providers/<idp_name>/protocols/openid/auth">
           AuthType oauth20
           Require valid-user
       </Location>
   
       <LocationMatch "/v3/auth/OS-FEDERATION/websso/openid">
           AuthType "openid-connect"
           Require valid-user
       </LocationMatch>

   Copy code to clipboard
   • Replace <my_client_id> with your client ID to use for the OpenID Connect provider handshake. You must get this from your SSO administrator.
   • Replace <my_client_secret> with the client secret to use for the OpenID Connect provider handshake. You must get this from your SSO administrator after providing your redirect URLs.
   • Replace <crypto_pass> with a secure passphrase to use when encrypting data for the OpenID Connect handshake. This is a user-defined value.
   • Replace <metadata_url> with the URL that points to your OpenID Connect provider metadata. Use the format: "https://<FQDN>/realms/<realm>/.well-known/openid-configuration. The SSO administrator will provide the requisite <FQDN> and organization-specific <realm> name for your OpenID provider.
   • Replace <https://my_oauth_introspection_endpoint> with the value provided by the SSO administrator.

   • Replace <idp_name> with your chosen string that creates unique redirect URL, for example, kcipaIDP. This value must be replaced for the keystoneFederationIdentityProviderName parameter and the LocationMatch and Location directive arguments.

     Important

     The full value for the OIDCRedirectURI parameter must end in a trailing /.

5. Create the secret:

   $ oc create -f keystone-httpd-override.yaml

   Copy code to clipboard

6. Get the URL for the OpenStack Dashboard:

   $ oc get horizons.horizon.openstack.org -o json | jq -r '.items[0].status.endpoint'

   Copy code to clipboard

7. Edit the keystone section of the OpenStackControlPlane CR file and add the secret:

   keystone:
     template:
       customServiceConfig: |
         [federation]
         trusted_dashboard=<horizon_endpoint>/dashboard/auth/websso/
         [openid]
         remote_id_attribute=HTTP_OIDC_ISS
         [auth]
         methods = password,token,oauth1,mapped,application_credential,openid
       httpdCustomization:
         customConfigSecret: keystone-httpd-override

   Copy code to clipboard
   • Replace <horizon_endpoint> with the value you retrieved in step 6.
   • Remove external from the methods = comma delimited list.
   • Add the httpdCustomization.customConfigSecret parameter and set this value to the key created in the keystone-httpd-override.yaml CR file.

8. Edit the horizon section of the OpenStackControlPlane CR file to configure the OpenStack Dashboard (horizon):

   horizon:
     template:
       customServiceConfig: |
         # Point Horizon to the Keystone public endpoint
         OPENSTACK_KEYSTONE_URL = "<keystone_endpoint>/v3"
   
         # Enable WebSSO in Horizon
         WEBSSO_ENABLED = True
   
         # Provide login options in Horizon's dropdown menu
         WEBSSO_CHOICES = (
           ("credentials", _("Keystone Credentials")),
           ("OIDC", _("OpenID Connect")),
         )
   
         # Map Horizon's "OIDC" choice to the Keystone IDP and protocol
         WEBSSO_IDP_MAPPING = {
           "OIDC": ("<idp_name>", "openid"),
         }

   Copy code to clipboard
   • Replace <keystone_endpoint> with the value you retrieved in the first step.
   • Replace <idp_name> with your chosen string that creates a unique redirect URL, for example, kcipaIDP.

9. Update the control plane:

   $ oc apply -f openstack_control_plane.yaml

   Copy code to clipboard

Procedure

1. Create a federated domain:

   $ openstack domain create <federated_domain_name>

   Copy code to clipboard

   • Replace <federated_domain_name> with the name of the domain you are managing with your identity provider, for example, my_domain.

     For example:

     +-------------+----------------------------------+
     | Field       | Value                            |
     +-------------+----------------------------------+
     | description |                                  |
     | enabled     | True                             |
     | id          | b493634c9dbf4546a2d1988af181d7c9 |
     | name        | my_domain                        |
     | options     | {}                               |
     | tags        | []                               |
     +-------------+----------------------------------+

     Copy code to clipboard

2. Set up the federation identity provider:

   $ openstack identity provider create --remote-id https://<sso_fqdn>:9443/realms/<realm> --domain <federated_domain_name> <idp_name>

   Copy code to clipboard
   • Replace <sso_fqdn> with the fully qualified domain name for your SSO identity provider.
   • Replace <realm> with the SSO realm. The default realm is master.
   • Replace <federated_domain_name> with the name of the federated domain that you created in step 1, for example, my_domain.

   • Replace <idp_name> with the string that you chose when deploying SSO to create the unique redirect URL, for example, kcipaIDP.

     For example:

     +-------------------+-----------------------------------------------------+
     | Field             | Value                                               |
     +-------------------+-----------------------------------------------------+
     | authorization_ttl | None                                                |
     | description       | None                                                |
     | domain_id         | b493634c9dbf4546a2d1988af181d7c9                    |
     | enabled           | True                                                |
     | id                | kcipaIDP                                            |
     | remote_ids        | https://sso.fqdn.local:9443/realms/master           |
     +-------------------+-----------------------------------------------------+

     Copy code to clipboard

3. Create a mapping file that is unique to the identity needs of your cloud:

   $ cat > mapping.json << EOF
   [
   	{
       	"local": [
           	{
               	"user": {
                	"name": "{0}"
               	},
               	"group": {
                   	"domain": {
                    	"name": "<federated_domain_name>"
                   	},
                   	"name": "<federated_group_name>"
               	}
           	}
       	],
       	"remote": [
           	{
               	"type": "OIDC-preferred_username"
           	}
       	]
   	}
   ]
   EOF

   Copy code to clipboard
   • Replace <federated_domain_name> with the domain you created in step 1, for example, my_domain.
   • Replace <federated_group_name> with the name of the federated group that you create in a later step, for example, my_fed_group.

4. Use the mapping file to create the federation mapping rules for RHOSO:

   $ openstack mapping create --rules <mapping_file> <mapping_rules>

   Copy code to clipboard
   • Replace <mapping_file> with the name of the mapping file that you created in the previous step, for example, mapping.json.
   • Replace <mapping_rules> with the name of the mapping rules created from this file, for example, IPAmap.

5. Create a federated group:

   $ openstack group create --domain <federated_domain_name> <federated_group_name>

   Copy code to clipboard
   • Replace <federated_domain_name> with the name of the domain that you created in step 1, for example, my_domain.
   • Replace <federated_group_name> with the name of the federated group that have specified in the mapping file, for example, my_fed_group.

6. Create an Identity service (keystone) project:

   $ openstack project create --domain <federated_domain_name> <federated_project_name>

   Copy code to clipboard
   • Replace <federation_project_name> with the name of the Identity service project.

7. Add the Identity service federation group to a role:

   $ openstack role add --group <federated_group_name> --group-domain <federated_domain_name> --project <federated_project_name> --project-domain <federated_domain_name> member

   Copy code to clipboard

8. Create the OpenID federation protocol:

   $ openstack federation protocol create openid --mapping <mapping_rules> --identity-provider <idp_name>

   Copy code to clipboard
   • Replace <mapping_rules> with the name of the mapping rules you created from your mapping file, for example, IPAmap.
   • Replace <idp_name> with your chosen string that creates the unique redirect URL, for example, kcipaIDP.

Procedure

1. Choose a name to uniquely identify each IdP.

   In this example there are two IdPs, whose names are referenced as <idp_name_1> and <idp_name_2>.

2. Obtain the following settings from each IdP administrator:

   • The FQDN for each IdP that is referenced in this procedure as <fqdn_1> and <fqdn_2>.
   • The federation Realm Name for each IdP that is referenced in this procedure as <realm_name_1> and <realm_name_2>.
   • The Client ID for each IdP that is referenced in this procedure as <client_id_1> and <client_id_2>.
   • The Client Secret for each IdP that is referenced in this procedure as <client_secret_1> and <client_secret_2>.
   • The Provider Metadata URL for each IdP that is referenced in this procedure as <provider_metadata_url_1> and <provider_metadata_url_2>.

3. Retrieve the Identity service (keystone) public endpoint:

   $ oc get keystoneapis.keystone.openstack.org -o json | jq '.items[0].status.apiEndpoints.public'

   Copy code to clipboard

   This Identity service endpoint is referenced in this procedure as <keystone_url>.

4. Provide the IdP administrators with the following information:

   • Web origin:

     https://<keystone_url>

     Copy code to clipboard

   • Redirect URIs:

     https://<keystone_url>/v3/auth/OS-FEDERATION/websso/openid

     Copy code to clipboard

     Provide a URI for each IdP containing their unique IdP name, which must end in a trailing /. You must send each URI to their respective IdP administrator:

     https://<keystone_url>/v3/auth/OS-FEDERATION/identity_providers/<idp_name_1>/protocols/openid/websso/
     https://<keystone_url>/v3/auth/OS-FEDERATION/identity_providers/<idp_name_2>/protocols/openid/websso/

     Copy code to clipboard
   • Each federation client must have Implicit flow enabled and not Authorization code flow.

5. Create a custom resource (CR) file for a secret called keystone-httpd-override:

   apiVersion: v1
   kind: Secret
   metadata:
    name: keystone-httpd-override
    namespace: openstack
   type: Opaque
   stringData:
    federation.conf: |
      # Example OIDC directives for the *public* endpoint
      OIDCClaimPrefix "OIDC-"
      OIDCResponseType "id_token"
      OIDCScope "openid email profile"
      OIDCClaimDelimiter ";"
      OIDCPassUserInfoAs "claims"
      OIDCPassClaimsAs "both"
      OIDCCryptoPassphrase "<crypto_pass>"
      OIDCRedirectURI "<keystone_url>/v3/redirect_uri/"
                           OIDCMetadataDir "/var/lib/httpd/metadata"
      OIDCAuthRequestParams "prompt=login"
      <IfModule headers_module>
        <Location "/v3/local-logout/clear">
          Header always add Set-Cookie "mod_auth_openidc_session=deleted; Path=/; Max-Age=0; HttpOnly; Secure; SameSite=None"
        </Location>
      </IfModule>
   
      RewriteEngine On
   
      RewriteRule ^/v3/auth/OS-FEDERATION/identity_providers/(<idp_name_1>|<idp_name_2>)/protocols/openid/websso$ \
        /v3/local-logout/clear [R=302,L]
   
      RewriteRule ^/v3/local-logout/clear$ \
        /v3/auth/OS-FEDERATION/websso/openid [R=302,L,QSA,NE]
   
      <Location "/v3/auth/OS-FEDERATION/websso/openid">
        AuthType openid-connect
        Require  valid-user
      </Location>
   
      <Location "/v3/redirect_uri">
        AuthType openid-connect
        Require  valid-user
      </Location>

   Copy code to clipboard
   Important

   The full value of the OIDCRedirectURI parameter must end in a trailing /.

   • Replace <crypto_pass> with a user-defined passphrase to use when encrypting data for the OpenID Connect handshake.
   • Replace <keystone_url> with the Identity service endpoint value that you retrieved in step 3.
   • Replace <idp_name_1> and <idp_name_2> with their unique IdP names that you specified in step 1.

   • The following OIDC parameter and associated Apache configuration is designed to provide the most failsafe solution that supports the login of users from multiple IdPs. Consequently, previous sessions are not saved and users must reauthenticate themselves after they log out of the Dashboard:

      OIDCAuthRequestParams "prompt=login"
     
      <IfModule headers_module>
        <Location "/v3/local-logout/clear">
          Header always add Set-Cookie "mod_auth_openidc_session=deleted; Path=/; Max-Age=0; HttpOnly; Secure; SameSite=None"
        </Location>
      </IfModule>
       RewriteEngine On
       RewriteRule ^/v3/auth/OS-FEDERATION/identity_providers/(<idp_name_1>|<idp_name_2>)/protocols/openid/websso$ \
        /v3/local-logout/clear [R=302,L]
       RewriteRule ^/v3/local-logout/clear$ \
        /v3/auth/OS-FEDERATION/websso/openid [R=302,L,QSA,NE]

     Copy code to clipboard

     If users in your multiple federated IdP deployment do not belong to more than one IdP then you can allow users to reopen the Dashboard they have closed without providing any authentication. In this case, you must remove this OIDC parameter and provide a different Apache LocationMatch configuration to save the previous sessions.

6. Create the keystone-httpd-override secret:

   $ oc create -f keystone-httpd-override.yaml

   Copy code to clipboard

7. Retrieve the URL for the Dashboard:

   $ oc get horizons.horizon.openstack.org -o json | jq -r '.items[0].status.endpoint'

   Copy code to clipboard

8. Use the following Ansible playbook to create a secret called federation-realm-data:

   - name: Download realm1 OpenID configuration
     ansible.builtin.uri:
       url: "<provider_metadata_url_1>"
       method: GET
       return_content: true
       validate_certs: false
     register: openid_wellknown_config1
   
   - name: Download realm2 OpenID configuration
     ansible.builtin.uri:
       url: "<provider_metadata_url_2>"
       method: GET
       return_content: true
       validate_certs: false
     register: openid_wellknown_config2
   
   - name: Set federation_config_items
     ansible.builtin.set_fact:
       federation_config_items:
         - filename: "<fqdn_1>%2Fauth%2Frealms%2F<realm_name_1>.conf"
           contents: |
             {
               "scope" : "openid email profile"
             }
         - filename: "<fqdn_1>%2Fauth%2Frealms%2F<realm_name_1>.client"
           contents: "{{ {'client_id': <client_id_1>, 'client_secret': <client_secret_1> } | to_json }}"
         - filename: "<fqdn_1>%2Fauth%2Frealms%2F<realm_name_1>.provider"
           contents: |
             {{ openid_wellknown_config1.content }}
         - filename: "<fqdn_2>%2Fauth%2Frealms%2F<realm_name_2>.conf"
           contents: |
             {
               "scope" : "openid email profile"
             }
         - filename: "<fqdn_2>%2Fauth%2Frealms%2F<realm_name_2>.client"
           contents: "{{ {'client_id': <client_id_2>, 'client_secret': <client_secret_2>} | to_json }}"
         - filename: "<fqdn_2>%2Fauth%2Frealms%2F<realm_name_2>.provider"
           contents: |
             {{ openid_wellknown_config2.content }}
   - name: Generate the final federation_config.json string (as a dictionary)
     ansible.builtin.set_fact:
       _raw_federation_config_json_value: |
         {
         {% for item in federation_config_items %}
           "{{ item.filename }}": {{ item.contents }}{% if not loop.last %},{% endif %}
         {% endfor %}
         }
   - name: Final JSON string for Secret stringData
     ansible.builtin.set_fact:
       federation_config_json_string: "{{ _raw_federation_config_json_value }}"
   
   - name: Create a Kubernetes Secret with federation metadata
     kubernetes.core.k8s:
       state: present
       definition:
         apiVersion: v1
         kind: Secret
         type: Opaque
         metadata:
           name: federation-realm-data
           namespace: openstack
         stringData:
           federation-config.json: "{{ federation_config_json_string }}"

   Copy code to clipboard
   • Replace the IdP variables with the values you obtained from the IdP administrators in step 2.
9. Open your OpenStackControlPlane custom resource (CR) file, openstack_control_plane.yaml, on your workstation.

10. Edit the keystone section of the OpenStackControlPlane CR:

    keystone:
     template:
       customServiceConfig: |
         [federation]
         trusted_dashboard=<horizon_endpoint>/dashboard/auth/websso/
         [openid]
         remote_id_attribute=HTTP_OIDC_ISS
         [auth]
         methods = password,token,oauth1,mapped,application_credential,openid
       httpdCustomization:
         customConfigSecret: keystone-httpd-override
         federatedRealmConfig: federation-realm-data

    Copy code to clipboard
    • Replace <horizon_endpoint> with the Dashboard URL you retrieved in step 7.
    • Remove external from the methods = comma delimited list.
    • Add the httpdCustomization.customConfigSecret parameter and set this value to the key created in the keystone-httpd-override.yaml CR file in step 5.
    • Add the httpdCustomization.federatedRealmConfig parameter and set this value to the federation-realm-data secret created by the Ansible Playbook in step 8.

11. Edit the horizon section of the OpenStackControlPlane CR:

    horizon:
     template:
     customServiceConfig: |
       # Point horizon to the keystone public endpoint
       OPENSTACK_KEYSTONE_URL = "<keystone_endpoint>/v3"
    
       # Enable WebSSO in horizon
       WEBSSO_ENABLED = True
    
       # Provide login options in the horizon dropdown menu
       WEBSSO_CHOICES = (
         ("credentials", _("Keystone Credentials")),
         ("OIDC1", _("OpenID Connect IdP1")),
         ("OIDC2", _("OpenID Connect IdP2")),
       )
    
       # Map the "OIDC" choice of horizon to the keystone IDP and protocol
       WEBSSO_IDP_MAPPING = {
         "OIDC1": ("<idp_name1>", "openid"),
         "OIDC2": ("<idp_name2>", "openid"),
       }

    Copy code to clipboard

12. Update the control plane:

    $ oc apply -f openstack_control_plane.yaml

    Copy code to clipboard

1. The interdependence between the workload and the central regions requires that every region must provide the following unique identifications for a successful SKMO deployment:

   • A unique namespace to differentiate the RHOSO deployment of each region. For more information, see Create a unique namespace for each workload region.

     Important

     The RHOSO deployment namespace forms part of the DNS name for each OpenStack service. If you do not use different RHOSO namespaces for every region, conflicts occur between services in your different regions.

   • A unique region name defined by the spec of their Identity (keystone) service in the OpenStackControlPlane custom resource (CR). For more information, see Modify the deployment of each workload region.

     Note

     When the central Red Hat OpenStack Services on OpenShift (RHOSO) region is deployed, this region is called regionOne by default. If you use a workload region naming convention, then you can rename the region name of the central region to make it more easily identifiable. For more information, see Rename the central region.

   • The OpenStack services that communicate with the Identity service must use uniquely named Identity service users to simplify the task of managing their individual credentials. For more information, see Modify the deployment of each workload region.

     Warning

     If you do not specify unique Identity service user names for all the OpenStack services that communicate with the Identity service, then changing the password for a service user disrupts this service in all the workload regions that use this same user - unless you schedule a maintenance window for all of these regions first.

2. The interdependence between the workload and the central regions imposes the following restrictions that must be met by the logical networking topology of your DNS configuration:

   • Each workload region must resolve the DNS name of the Identity service in the central region and access it.
   • The data plane nodes in each workload region must resolve the DNS name of the Identity service in the central region and access it.
   • After deploying the workload regions, the Dashboard (horizon) service in the central region must resolve the DNS names in the service catalog of every workload region and access them.

3. The interdependence between the Identity services in each workload region and the Identity service of the central region changes how the service-to-service communication for the workload regions are routed. For more information, see Create the public and private endpoints of each workload region.

   In a normal OpenStack deployment like the central region, the Identity service has both a public and internal endpoint. These endpoints exist in separate networks to keep the internal service-to-service communication separate from public traffic. However, the workload regions are forced to send all of their internal service-to-service communication traffic to the public endpoint and therefore the public network of the central region. Even though the internal service-to-service communication traffic of the workload regions is encrypted, it is more vulnerable to DDOS attacks because it is not isolated on a separate internal network making it easier for external attackers to intercept these messages.

4. The barbican-keystone-listener service requires access to the RabbitMQ message queue so that when a project is deleted by the Identity service (keystone), it can tell the Key Manager service (barbican) to clean up the related secrets and the other artifacts that it manages.

   In an SKMO deployment, the RabbitMQ message queue of the central region and not the workload regions contain the necessary Identity service (keystone) messages. For this reason, the barbican-keystone-listener services in the workload regions cannot know when projects are deleted so that the Key Manager service cannot clean them up. Therefore you must implement a third-party application like Scupper and configure your SKMO deployment to allow the barbican-keystone-listener services in the workload regions to access the RabbitMQ message queue in the central region to clean up deleted projects.

Procedure

1. Deploy the central region called regionOne by default, unless you rename it. For more information, see Rename the central region.

   The deployment of the central region does not require a data plane.

2. Create the default administrator Identity service user for each workload region in the central region. These Identity service users must be granted the admin role in the admin project of the central region. For more information, see Create the default administrator user for each workload region.

3. Create the catalog entries for the public and private endpoints of the Identity service in each workload region by using the Identity service in the central region. For more information, see Create the public and private endpoints of each workload region.

   Note

   Both the public and private endpoints of the Identity service in each workload region point to the public Identity service endpoint in the central region.

4. Modify and deploy each workload region. An important part of this deployment modification involves creating a unique region name and unique Identity service users for each workload region. For more information, see Modify the deployment of each workload region.
5. After you deploy a workload region of a multi-region Red Hat OpenStack Services on OpenShift (RHOSO) Single Keystone Multiple OpenStacks (SKMO) deployment you must configure the deployed central region to trust this workload region. For more information, see Configure the central region to trust a deployed workload region.

Procedure

1. In the central region, set the default RHOSO namespace:

   $ oc project <central-region-namespace>

   Copy code to clipboard
   • Replace <central-region-namespace> with the name of the unique namespace for the central region, for example openstack.

2. Edit the OpenStackControlPlane CR of your central region on your workstation:

   $ oc edit openstackcontrolplane <name>

   Copy code to clipboard
   • Replace <name> with the name of your YAML OpenStackControlPlane CR. You can use the following command to retrieve this name: oc get openstackcontrolplane.

3. Configure the region parameter of the Identity service (keystone):

   ...
     spec:
       ...
       keystone:
         ...
         template:
           ...
           region: <central-region-name>

   Copy code to clipboard
   • Replace <central-region-name> with the region name for your central region.
4. Save and close the editor to apply this change.

5. Wait for the deployment of the control plane to reach the Ready status:

   $ oc wait openstackcontrolplane <name> --for=condition=Ready --timeout=600s

   Copy code to clipboard

Procedure

1. In the central region, set the default RHOSO namespace:

   $ oc project <central-region-namespace>

   Copy code to clipboard
   • Replace <central-region-namespace> with the name of the unique namespace for the central region, for example openstack.

2. Access the remote shell for the OpenStackClient pod from your workstation to run OpenStack CLI commands:

   $ oc rsh openstackclient

   Copy code to clipboard

3. Create the default administrator user for each workload region:

   $ openstack user create --domain Default --project <central-region-admin-project> --project-domain Default --password <workload-region-admin-password> <workload-region-admin-name>

   Copy code to clipboard
   • Replace <central-region-admin-project> with the name of the admin project in the central region that is admin by default.

   • Replace <workload-region-admin-password> with the password of the default administrator user of each`workload` region.

     Note

     Set this password as the value of the AdminPassword: parameter of the Secret custom resource (CR) file that you must create to provide secure access to the RHOSO service pods when you deploy each workload region.

   • Replace <workload-region-admin-name> with the name of the default administrator user of each workload region, for example admin-two.

4. Add the following roles to each default workload region administrator user:

   $ openstack role add --project admin --project-domain Default --user <workload-region-admin-name> --user-domain Default admin
   $ openstack role add --system all --user <workload-region-admin-name> --user-domain Default admin

   Copy code to clipboard

Procedure

1. In the central region, set the default RHOSO namespace:

   $ oc project <central-region-namespace>

   Copy code to clipboard
   • Replace <central-region-namespace> with the name of the unique namespace for the central region, for example openstack.

2. Access the remote shell for the OpenStackClient pod from your workstation to run OpenStack CLI commands:

   $ oc rsh openstackclient

   Copy code to clipboard

3. Obtain the public Identity service endpoint of the central region:

   $ openstack endpoint list --region <central-region-name> --service keystone --interface public

   Copy code to clipboard
   • Replace <central-region-name> with the name of your central region, which is regionOne by default.
4. Copy this URL that is referenced in this procedure as <central-region-public-keystone-url>.

5. Create the public and private endpoints of each workload region to specify the public Identity service endpoint of the central region:

   $ openstack endpoint create --region <workload-region-name> keystone public <central-region-public-keystone-url>
   $ openstack endpoint create --region <workload-region-name> keystone internal <central-region-public-keystone-url>

   Copy code to clipboard

   • Replace <workload-region-name> with the name of the required workload region, for example regionTwo.

     This creates the catalog entries for the public and private endpoints of the Identity service in each workload region in the Identity service catalog of the central region.

Procedure

1. Create a unique RHOSO deployment namespace for each workload region to differentiate these RHOSO deployments from each other. For more information, see Create a unique namespace for each workload region.
2. Specify the password of the administrator user that you manually created for each workload region as the value of the AdminPassword: parameter when you create the Secret custom resource (CR) to provide secure access to the RHOSO service pods in each workload region. For more information, see Create the default administrator user for each workload region and Provide secure access to RHOSO services in the Deploying Red Hat OpenStack Services on OpenShift guide.
3. Obtain the CA certificate from the central region and add it to a secret in each workload region so it can be trusted. For more information, see Configure each workload region to trust the central region.

4. Create a modified OpenStackControlPlane custom resource (CR) for each workload region. For more information, see Modify the control plane CR of each workload region.

   This involves defining a unique region name defined in the spec of the Identity (keystone) service for each workload region and creating unique Identity service users.

Procedure

1. In the deployed workload region, set the default RHOSO namespace:

   $ oc project <workload-region-namespace>

   Copy code to clipboard
   • Replace <workload-region-namespace> with the name of the unique namespace for this workload region, for example openstack-two.

2. Obtain the CA certificate of this workload region and extract it into a file, for example regionTwo-ca.crt:

   Note

   To decode the certificate before creating the output .crt file add | base64 -d to this command.

   $ oc get secret rootca-public -o yaml | yq '.data."ca.crt"' > regionTwo-ca.crt

   Copy code to clipboard
3. Copy the regionTwo-ca.crt file to the deployed central region.

4. In the central region, set the default RHOSO namespace:

   $ oc project <central-region-namespace>

   Copy code to clipboard
   • Replace <central-region-namespace> with the name of the unique namespace for the central region, for example openstack.

5. Edit your OpenStackControlPlane CR of the central region:

   $ oc edit openstackcontrolplane <name>

   Copy code to clipboard
   • Replace <name> with the name of your YAML OpenStackControlPlane CR. You can use the following command to retrieve this name: oc get openstackcontrolplane.

6. If your OpenStackControlPlane CR in the central region contains the spec.tls.caBundleSecretName parameter:

   1. Obtain the name of the secret that contains the PEM-formatted bundle containing all the other custom CA certificates in chains of trust if applicable that the central region trusts, for example custom-ca-certs:

      ...
        spec:
          ...
          tls:
            ...
            caBundleSecretName: custom-ca-certs

      Copy code to clipboard

   2. Edit the specified secret, in this example custom-ca-certs

      $ oc edit secret custom-ca-certs

      Copy code to clipboard
   3. Append the contents of the CA certificate of the deployed workload region, for example regionTwo-ca.crt to the PEM-formatted bundle containing all the other custom CA certificates that the central region trusts.
   4. Save and exit the editor to automatically apply the changes to the secret, in this example custom-ca-certs.

7. If your OpenStackControlPlane CR in the central region does not contain the spec.tls.caBundleSecretName parameter:

   1. Create a PEM-formatted bundle, for example custom-ca-certs.pem that includes the contents of this regionTwo-ca.crt file.

   2. Create a manifest file for the secret in the central region that specifies the contents of the custom-ca-certs.pem bundle created in the previous step. In this example the manifest file is called custom-ca-certs.yaml and the secret is called custom-ca-certs:

      apiVersion: v1
      data:
        custom-ca-certs.pem: <contents-of-PEM-bundle>
      kind: Secret
      metadata:
        annotations:
        name: custom-ca-certs
        namespace: <namespace>
      type: Opaque

      Copy code to clipboard
      • Replace <namespace> with the namespace of the central region, in this example openstack.
      • Replace <contents-of-PEM-bundle> with the base64 encoded string of the contents of the PEM-formatted bundle you created called custom-ca-certs.pem that includes the CA certificate from regionTwo. You can get this base64 encoded string by using the following command: cat custom-ca-certs.pem | base64 -w0.

3. Create the secret in the central region from the manifest file. In this example the manifest file is called custom-ca-certs.yaml:

   $ oc apply -f custom-ca-certs.yaml

   Copy code to clipboard

4. Edit your OpenStackControlPlane CR of the central region:

   $ oc edit openstackcontrolplane <name>

   Copy code to clipboard
   • Replace <name> with the name of your YAML OpenStackControlPlane CR. You can use the following command to retrieve this name: oc get openstackcontrolplane.

5. Add the secret that you have created, in this example custom-ca-certs:

   ...
     spec:
       ...
       tls:
         ...
         caBundleSecretName: custom-ca-certs

   Copy code to clipboard
6. Save and close the editor to automatically apply this change.

7. Wait for the deployment of the control plane of the central region to reach the Ready status:

   $ oc wait openstackcontrolplane <name> --for=condition=Ready --timeout=600s

   Copy code to clipboard