Tutorials

OpenShift Dedicated 4

OpenShift Dedicated tutorials

Red Hat OpenShift Documentation Team

Abstract

Tutorials for managing OpenShift Dedicated clusters.

Chapter 1. Tutorial: Updating component routes with custom domains and TLS certificates

Change the hostname and Transport Layer Security (TLS) certificate of the web console, OAuth server, and Downloads component routes to use custom domains that align with your organization’s branding and security requirements.

Important

Red Hat experts authored this content, but it has not yet been tested on every supported configuration.

1.1. Setting up your environment for component route updates

Log in to your cluster as an admin user and configure environment variables to streamline the component route update workflow.

Prerequisites

  • You have installed Red Hat OpenShift Cluster Manager command-line interface (CLI) (ocm) version 1.0.5 or higher.
  • You have installed gcloud CLI.
  • You have created an OpenShift Dedicated on Google Cloud cluster version 4.14 or higher.
  • You have installed OpenShift CLI (oc).
  • You have installed jq CLI.
  • You have confirmed that you have access to the cluster as a user with the cluster-admin role.
  • You have installed OpenSSL (for generating the demonstration SSL/TLS certificates).

Procedure

  1. Log in to your cluster using an account with cluster-admin privileges.

  2. Configure an environment variable for your cluster name: 

    $ export CLUSTER_NAME=$(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}"  | sed 's/-[a-z0-9]\{5\}$//')
$ export CLUSTER_ID=$(oc get clusterversion version -o jsonpath='{.spec.clusterID}')

Verification

  • Ensure the environment variable is set correctly: 

    $ echo "Cluster Name: ${CLUSTER_NAME}"
$ echo "Cluster ID: ${CLUSTER_ID}"

    Example output

    Cluster Name: my-osd-cluster
Cluster ID: 12a3b456-78cd-90ef-1234-56789abcdef0

1.2. Finding the current component routes

Find the base hostname of your cluster routes to verify the default component route configuration.

Procedure

  1. Verify that you can reach the component routes on their default hostnames. You can find the hostnames by querying the lists of routes in the openshift-console and openshift-authentication projects. 

    $ oc get routes -n openshift-console
$ oc get routes -n openshift-authentication

    Example output

    NAME        HOST/PORT                                                                          PATH       SERVICES    PORT    TERMINATION          WILDCARD
console     console-openshift-console.apps.my-example-cluster-gcp.<cluster_id>.openshiftapps.com    ... 1 more  console    https   reencrypt/Redirect   None
downloads   downloads-openshift-console.apps.my-example-cluster-gcp.<cluster_id>.openshiftapps.com  ... 1 more  downloads  http    edge/Redirect        None
NAME              HOST/PORT                                                             PATH        SERVICES          PORT   TERMINATION            WILDCARD
oauth-openshift   oauth-openshift.apps.my-example-cluster-gcp.<cluster_id>.openshiftapps.com ... 1 more  oauth-openshift   6443   passthrough/Redirect   None

    By running these commands you can see that the default component routes for your cluster are:

    • console-openshift-console.apps.my-example-cluster-gcp.<cluster_id>.openshiftapps.com for Console
    • downloads-openshift-console.apps.my-example-cluster-gcp.<cluster_id>.openshiftapps.com for Downloads

    • oauth-openshift.apps.my-example-cluster-gcp.<cluster_id>.openshiftapps.com for OAuth

      From this output you can see that your base hostname is <cluster_id>.openshiftapps.com.

  2. Get the ID of the default ingress by running the following command: 

    $ export INGRESS_ID=$(ocm list ingresses -c ${CLUSTER_NAME} | awk '$4 == "true" {print $1}')

  3. Ensure all fields output correctly before moving to the next section: 

    $ echo "Ingress ID: ${INGRESS_ID}"

    Example output

    Ingress ID: r3l6

  4. Use the ocm edit ingress command to change the hostname of each service and add a TLS certificate for all of your component routes. This excerpt of the command-line help for the ocm edit ingress command shows the relevant parameters: 

    $ ocm edit ingress -h
Edit a cluster ingress for a cluster. Usage:
  ocm edit ingress ID [flags]
  [...]
  --component-routes string                Component routes settings. Available keys [oauth, console, downloads]. For each key a pair of hostname and tlsSecretRef is expected to be supplied. Format should be a comma separate list 'oauth: hostname=example-hostname;tlsSecretRef=example-secret-ref,downloads:...'

1.3. Creating TLS certificates for each component route

Create three self-signed certificates, one for each component route. Trust them on your system so you can open each new hostname in a browser.

Warning

Use this flow for learning only, not for production. For live systems, request valid certificates from your certificate authority (CA).

Important

Use one certificate per route to prevent issues with HTTP/2 connection coalescing. Wildcard certificates and subject alternative names (SAN) certificates are not supported.

This example uses the following custom component routes:

  • console.example.com for Console
  • downloads.console.example.com for Downloads
  • oauth.console.example.com for OAuth

Procedure

  • For each route, run the example openssl commands. Set -subj to that route’s domain name:

    Example output:

    $ openssl req -newkey rsa:2048 -new -nodes -x509 -days 365 -keyout key-console.pem -out cert-console.pem -subj "/CN=console.example.com"
$ openssl req -newkey rsa:2048 -new -nodes -x509 -days 365 -keyout key-downloads.pem -out cert-downloads.pem -subj "/CN=downloads.console.example.com"
$ openssl req -newkey rsa:2048 -new -nodes -x509 -days 365 -keyout key-oauth.pem -out cert-oauth.pem -subj "/CN=oauth.console.example.com"

Verification

  • Check that the .pem certificate and key files exist: 

    $ ls -1 *.pem

    Example output

    cert-console.pem
cert-downloads.pem
cert-oauth.pem
key-console.pem
key-downloads.pem
key-oauth.pem

1.4. Adding the certificates to the cluster as secrets

Add the Transport Layer Security (TLS) certificates to your cluster as secrets in the openshift-config namespace to reference them when updating component routes.

Procedure

  • Create three TLS secrets in the openshift-config namespace.

    These become your secret reference when you update the component routes. 

    $ oc create secret tls console-tls --cert=cert-console.pem --key=key-console.pem -n openshift-config
$ oc create secret tls downloads-tls --cert=cert-downloads.pem --key=key-downloads.pem -n openshift-config
$ oc create secret tls oauth-tls --cert=cert-oauth.pem --key=key-oauth.pem -n openshift-config

Verification

  • Verify that the TLS secrets were created: 

    $ oc get secrets -n openshift-config | grep -E 'console-tls|downloads-tls|oauth-tls'

    The output shows the three TLS secrets.

1.5. Finding the load balancer IP address

Find the load balancer internet protocol (IP) address of your cluster to create domain name system (DNS) records for the component route hostnames.

Procedure

  1. Retrieve the IP address of the load balancer by running the following command, using the namespace for the load balancer: 

    $ oc get svc -n <namespace>

    The load balancer IP of the load balancer is the EXTERNAL-IP associated with the router-default service in the openshift-ingress namespace.

    Example output

    $ oc get svc -n openshift-ingress
NAME            TYPE          CLUSTER-IP     EXTERNAL-IP        PORT(S)                     AGE
router-default  LoadBalancer  172.30.237.88  203.0.113.10      80:31175/TCP,443:31554/TCP  76d

    In this example, the load balancer IP is 203.0.113.10.

  2. Save this value for later, as you need it to configure DNS records for your new component route hostnames.
  3. Create an A record in your DNS settings, pointing the domain to the IP address of the load balancer for router-default.

1.6. Updating the component routes and TLS certificates

Use the Red Hat OpenShift Cluster Manager CLI to apply your custom hostnames and TLS certificates to the component routes.

Procedure

  1. Use the ocm edit ingress command to update your default ingress route with the new base domain and the secret reference associated with it, and update the hostnames for each component route. 

    $ ocm edit ingress -c ${CLUSTER_NAME} ${INGRESS_ID} --component-routes 'console: hostname=console.my-new-domain.dev;tlsSecretRef=console-tls,downloads: hostname=downloads.my-new-domain.dev;tlsSecretRef=downloads-tls,oauth: hostname=oauth.my-new-domain.dev;tlsSecretRef=oauth-tls'
    Note

    You can also edit only a subset of the component routes by leaving the component routes you do not want to change set to an empty string. For example, if you only want to change the Console and OAuth server hostnames and TLS certificates, you would run the following command: 

    $ ocm edit ingress -c ${CLUSTER_NAME} ${INGRESS_ID} --component-routes 'console: hostname=console.my-new-domain.dev;tlsSecretRef=console-tls,downloads: hostname="";tlsSecretRef="", oauth: hostname=oauth.my-new-domain.dev;tlsSecretRef=oauth-tls'

  2. Run the ocm list ingress command to verify your changes: 

    $ ocm list ingress -c ${CLUSTER_NAME} -ojson | jq ".[] | select(.id == \"${INGRESS_ID}\") | .component_routes"

    Example output

    {
  "console": {
    "kind": "ComponentRoute",
    "hostname": "console.my-new-domain.dev",
    "tls_secret_ref": "console-tls"
  },
  "downloads": {
    "kind": "ComponentRoute",
    "hostname": "downloads.my-new-domain.dev",
    "tls_secret_ref": "downloads-tls"
  },
  "oauth": {
    "kind": "ComponentRoute",
    "hostname": "oauth.my-new-domain.dev",
    "tls_secret_ref": "oauth-tls"
  }
}

1.7. Resetting routes to defaults

Reset the routes, use default hostnames, and remove custom TLS certs.

Procedure

  • Reset your routes by running the following command: 

    $ ocm edit ingress -c ${CLUSTER_NAME} ${INGRESS_ID} --component-routes 'console: hostname="";tlsSecretRef="",downloads: hostname="";tlsSecretRef="", oauth: hostname="";tlsSecretRef=""'

Verification

  • Check that hostnames and TLS cert refs use defaults: 

    $ ocm get /api/clusters_mgmt/v1/clusters/${CLUSTER_ID}/ingresses/${INGRESS_ID} | jq .component_routes

    Example output

    {
  "console": {
    "hostname": "console.my-new-domain.dev",
    "tls_secret_ref": "console-tls"
  },
  "downloads": {
    "hostname": "downloads.my-new-domain.dev",
    "tls_secret_ref": "downloads-tls"
  },
  "oauth": {
    "hostname": "oauth.my-new-domain.dev",
    "tls_secret_ref": "oauth-tls"
  }
}

    The output shows empty hostname and tls_secret_ref for each route.

1.8. Additional resources

Chapter 2. Tutorial: Limit egress with Google Cloud Next Generation Firewall

Implement egress restrictions for OpenShift Dedicated on Google Cloud by using Next Generation Firewall (NGFW), which allows fully qualified domain name (FQDN)-based firewall rules required for OpenShift Dedicated external endpoints.

Important

This content is authored by Red Hat experts but has not yet been tested on every supported configuration.

2.1. Setting up your environment

Set environment variables so each command in this tutorial uses the same values for OpenShift Dedicated on Google Cloud with your firewall rules.

Prerequisites

  • You have the Google Cloud command-line interface (CLI) (gcloud) installed.
  • You are logged into the Google Cloud CLI and have selected the Google Cloud project where you plan to deploy OpenShift Dedicated.

  • You have the minimum necessary permissions in Google Cloud, including:

    • Compute Network Admin
    • Domain Name System (DNS) Administrator

  • You enabled the required Google Cloud services:

    • networksecurity.googleapis.com
    • networkservices.googleapis.com

    • servicenetworking.googleapis.com

      You can enable these services by running the following commands: 

      $ gcloud services enable networksecurity.googleapis.com
$ gcloud services enable networkservices.googleapis.com
$ gcloud services enable servicenetworking.googleapis.com

Procedure

  • Run this command to set the environment variables: 

    $ export project_id=$(gcloud config list --format="value(core.project)")
$ export region=us-east1
$ export prefix=osd-ngfw
$ export service_cidr="172.30.0.0/16"
$ export machine_cidr="10.0.0.0/22"
$ export pod_cidr="10.128.0.0/14"

    This example sets the region to us-east1 and the name prefix to osd-ngfw. The service and pod networks use the default Classless Inter-Domain Routing (CIDR) ranges in the export list. You add subnet ranges later in this tutorial. The machine CIDR must fit inside those subnet ranges. Change the exports to match your project.

2.2. Creating the VPC and subnets

Create the Virtual Private Cloud (VPC) and subnets required for deploying Google Cloud Next Generation Firewall (NGFW) with OpenShift Dedicated.

Procedure

  1. Create the VPC by running the following command: 

    $ gcloud compute networks create ${prefix}-vpc --subnet-mode=custom

  2. Create the worker subnets by running the following command: 

    $ gcloud compute networks subnets create ${prefix}-worker \
    --range=10.0.2.0/23 \
    --network=${prefix}-vpc \
    --region=${region} \
    --enable-private-ip-google-access

  3. Create the control plane subnets by running the following command: 

    $ gcloud compute networks subnets create ${prefix}-control-plane \
    --range=10.0.0.0/25 \
    --network=${prefix}-vpc \
    --region=${region} \
    --enable-private-ip-google-access

  4. Create the Private Service Connect (PSC) subnets by running the following command: 

    $ gcloud compute networks subnets create ${prefix}-psc \
    --network=${prefix}-vpc \
    --region=${region} \
    --stack-type=IPV4_ONLY \
    --range=10.0.0.128/29 \
    --purpose=PRIVATE_SERVICE_CONNECT

    These examples use the subnet ranges of 10.0.2.0/23 for the worker subnet, 10.0.0.0/25 for the control plane subnet, and 10.0.0.128/29 for the PSC subnet. Modify the parameters to meet your needs. Ensure the parameter values are contained within the machine CIDR you set earlier in this tutorial.

Verification

  • Verify the VPC and subnets were created by running the following command: 

    $ gcloud compute networks subnets list --network=${prefix}-vpc

    The output shows the three subnets you created with their internet protocol (IP) ranges and regions.

2.3. Deploying a global firewall policy

Create a global network firewall policy. Attach it to your VPC so you can control traffic that leaves your OpenShift Dedicated cluster.

Procedure

  1. Run this command to create a global network firewall policy: 

    $ gcloud compute network-firewall-policies create \
    ${prefix} \
    --description "OpenShift Dedicated Egress Firewall" \
    --global

  2. Run this command to attach the new policy to the VPC you created earlier: 

    $ gcloud compute network-firewall-policies associations create \
      --name ${prefix}-vpc-association \
      --firewall-policy ${prefix} \
      --network ${prefix}-vpc \
      --global-firewall-policy

Verification

  • Run this command to check that the policy exists and is attached to your VPC: 

    $ gcloud compute network-firewall-policies describe ${prefix} --global

    The output lists the policy and its link to your VPC.

2.4. Creating a Cloud Router and Cloud network address translation

Create a Cloud Router and Cloud network address translation (NAT). Private VMs can use the internet while their private IP addresses stay hidden.

Procedure

  1. Reserve an IP address for Cloud NAT by running the following command: 

    $ gcloud compute addresses create ${prefix}-${region}-cloudnatip \
    --region=${region}

  2. Create a Cloud Router by running the following command: 

    $ gcloud compute routers create ${prefix}-router \
    --region=${region} \
    --network=${prefix}-vpc

  3. Create a Cloud NAT by running the following command: 

    $ gcloud compute routers nats create ${prefix}-cloudnat-${region} \
    --router=${prefix}-router --router-region ${region} \
    --nat-all-subnet-ip-ranges \
    --nat-external-ip-pool=${prefix}-${region}-cloudnatip

Verification

  • Check that the Cloud Router and NAT gateway exist by running the following command: 

    $ gcloud compute routers describe ${prefix}-router --region=${region}

    The output lists the router and the NAT gateway you created.

2.5. Creating private Domain Name System records for private Google access

Create a private Domain Name System (DNS) zone to route Google application programming interface (API) traffic through the internal network of Google for faster and more secure connections.

Procedure

  1. Create a private DNS zone for the googleapis.com domain by running the following command: 

    $ gcloud dns managed-zones create ${prefix}-googleapis \
    --visibility=private \
    --networks=https://www.googleapis.com/compute/v1/projects/${project_id}/global/networks/${prefix}-vpc \
    --description="Private Google Access" \
    --dns-name=googleapis.com

  2. Begin a record set transaction by running the following command: 

    $ gcloud dns record-sets transaction start \
    --zone=${prefix}-googleapis

  3. Stage the DNS records for Google APIs under the googleapis.com domain by running the following commands: 

    $ gcloud dns record-sets transaction add --name="*.googleapis.com." \
    --type=CNAME restricted.googleapis.com. \
    --zone=${prefix}-googleapis \
    --ttl=300
    $ gcloud dns record-sets transaction add 199.36.153.4 199.36.153.5 199.36.153.6 199.36.153.7 \
    --name=restricted.googleapis.com. \
    --type=A \
    --zone=${prefix}-googleapis \
    --ttl=300

  4. Apply the staged record set transaction you started above by running the following command: 

    $ gcloud dns record-sets transaction execute \
    --zone=$prefix-googleapis

Verification

  • Verify the private DNS zone and records were created by running the following command: 

    $ gcloud dns record-sets list --zone=${prefix}-googleapis

    The output shows the DNS zone with CNAME and A records for googleapis.com.

2.6. Creating the firewall rules

Create firewall rules for egress to private IP ranges and to the OpenShift Dedicated domains listed in this procedure. Egress to other external destinations does not match these rules and is not permitted.

Procedure

  1. Create a blanket allow rule for private IP (Request for Comments (RFC) 1918) address space by running the following command: 

    $ gcloud compute network-firewall-policies rules create 500 \
    --description "Allow egress to private IP ranges" \
    --action=allow \
    --firewall-policy=${prefix} \
    --global-firewall-policy \
    --direction=EGRESS \
    --layer4-configs all \
    --dest-ip-ranges=10.0.0.0/8,172.16.0.0/12,192.168.0.0/16

  2. Create an allow rule for HTTPS (tcp/443) domains required for OpenShift Dedicated by running the following command:

    Note

    If you receive an error "Cannot have rules with the same priorities", the rule already exists. You can verify it with: 

    $ gcloud compute network-firewall-policies rules describe 500 --firewall-policy=${prefix} --global-firewall-policy
$ gcloud compute network-firewall-policies rules describe 600 --firewall-policy=${prefix} --global-firewall-policy

    To re-create the rules, first delete them: 

    $ gcloud compute network-firewall-policies rules delete 500 --firewall-policy=${prefix} --global-firewall-policy
$ gcloud compute network-firewall-policies rules delete 600 --firewall-policy=${prefix} --global-firewall-policy
    $ gcloud compute network-firewall-policies rules create 600 \
    --description "Allow egress to OpenShift Dedicated required domains (tcp/443)" \
    --action=allow \
    --firewall-policy=${prefix} \
    --global-firewall-policy \
    --direction=EGRESS \
    --layer4-configs tcp:443 \
    --dest-fqdns accounts.google.com,pull.q1w2.quay.rhcloud.com,http-inputs-osdsecuritylogs.splunkcloud.com,nosnch.in,api.deadmanssnitch.com,events.pagerduty.com,api.pagerduty.com,api.openshift.com,mirror.openshift.com,observatorium.api.openshift.com,observatorium-mst.api.openshift.com,console.redhat.com,infogw.api.openshift.com,api.access.redhat.com,cert-api.access.redhat.com,catalog.redhat.com,sso.redhat.com,registry.connect.redhat.com,registry.access.redhat.com,cdn01.quay.io,cdn02.quay.io,cdn03.quay.io,cdn04.quay.io,cdn05.quay.io,cdn06.quay.io,cdn.quay.io,quay.io,registry.redhat.io,quayio-production-s3.s3.amazonaws.com
    Important

    The firewall blocks any traffic if you did not create any matching rules. To allow access to other resources, such as internal networks or other external endpoints, create additional rules with a priority of less than 1000. For more information on how to create firewall rules, see the Additional resources.

2.7. Cluster creation

Your OpenShift Dedicated cluster on Google Cloud uses the VPC, subnets, and firewall rules from this tutorial.

For detailed instructions on creating a cluster, see Creating a cluster on Google Cloud.

2.8. Cluster deletion

When you delete your cluster on Google Cloud, also clean up the network setup from this guide to prevent ongoing charges.

For detailed instructions on deleting a cluster, see Deleting an OpenShift Dedicated cluster on Google Cloud.

2.9. Cleaning up resources

Delete the Google Cloud networking infrastructure after deleting your cluster to prevent ongoing charges. The cluster deletion does not automatically remove virtual private cloud (VPC) networks, subnets, firewall policies, or domain name system (DNS) zones.

Procedure

  1. Authenticate by running the following command: 

    $ gcloud init

  2. Log in to your Google Cloud account by running the following command: 

    $ gcloud auth application-default login

  3. Log in to the OpenShift Cluster Manager CLI tool by running the following command: 

    $ ocm login --use-auth-code

    You can now clean up the resources you created as part of this tutorial. To respect resource dependencies, delete them in the reverse order of their creation.

  4. Delete the association of the firewall policy with the VPC by running the following command: 

    $ gcloud compute network-firewall-policies associations delete \
      --name network-${prefix}-vpc \
      --firewall-policy=${prefix} \
      --global-firewall-policy \
      --project=${project_id}

  5. Delete the global network firewall policy by running the following command: 

    $ gcloud compute network-firewall-policies delete ${prefix} --global --project=${project_id}

  6. List and delete all user-defined DNS records from the Private DNS zone: 

    $ gcloud dns record-sets list \
    --project=${project_id} \
    --zone=${prefix}-googleapis \
    --filter="type!=NS AND type!=SOA" \
    --format="value(name,type)" | while read name type; do
  gcloud dns record-sets delete "$name" \
    --project=${project_id} \
    --zone=${prefix}-googleapis \
    --type="$type"
done

  7. Delete the Private DNS Zone by running the following command: 

    $ gcloud dns managed-zones delete ${prefix}-googleapis --project=${project_id}

  8. Delete the Cloud NAT gateway: 

    $ gcloud compute routers nats delete ${prefix}-cloudnat-${region} \
    --router=${prefix}-router \
    --router-region=${region} \
    --project=${project_id}

  9. Delete the Cloud Router by running the following command: 

    $ gcloud compute routers delete ${prefix}-router --region=${region} --project=${project_id}

  10. Delete the reserved IP address by running the following command: 

    $ gcloud compute addresses delete ${prefix}-${region}-cloudnatip --region=${region} --project=${project_id}

  11. Delete the worker subnet by running the following command: 

    $ gcloud compute networks subnets delete ${prefix}-worker --region=${region} --project=${project_id}

  12. Delete the control plane subnet by running the following command: 

    $ gcloud compute networks subnets delete ${prefix}-control-plane --region=${region} --project=${project_id}

  13. Delete the Private Service Connect (PSC) subnet by running the following command: 

    $ gcloud compute networks subnets delete ${prefix}-psc --region=${region} --project=${project_id}

  14. Delete the VPC by running the following command: 

    $ gcloud compute networks delete ${prefix}-vpc --project=${project_id}

2.10. Additional resources

Legal Notice

Copyright © Red Hat.
Except as otherwise noted below, the text of and illustrations in this documentation are licensed by Red Hat under the Creative Commons Attribution–Share Alike 3.0 Unported license . If you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, the Red Hat logo, JBoss, Hibernate, and RHCE are trademarks or registered trademarks of Red Hat, LLC. or its subsidiaries in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
XFS is a trademark or registered trademark of Hewlett Packard Enterprise Development LP or its subsidiaries in the United States and other countries.
The OpenStack® Word Mark and OpenStack logo are trademarks or registered trademarks of the Linux Foundation, used under license.
All other trademarks are the property of their respective owners.