Configuring External DNS for RHOAI 3.x on Bare Metal, OpenStack and Private Clouds

Updated

Overview of RHOAI 3.x Ingress Changes

In Red Hat OpenShift AI (RHOAI) version 3.x, the method for handling cluster ingress traffic has changed. RHOAI is transitioning away from using OpenShift Routes and is adopting the Gateway API.

When RHOAI is deployed, the following resources are automatically created on the cluster by the ingress operator:

  • Gateway Resource: This resource is responsible for managing all external traffic directed to RHOAI components, including the Dashboard and Workbenches.
  • LoadBalancer Service: The deployment of the Gateway automatically creates a new LoadBalancer Service. This Service is assigned a new, distinct external IP address for ingress traffic from the external load balancer.

IMPORTANT: This new LoadBalancer IP address is separate from any IP address previously used for OpenShift Routes.

NOTE: If the cluster is using a supported cloud load balancer, the ingress operator will request a new DNS A-record name (for example, 1234567.us-east-1.elb.amazonaws.com). That name then becomes the “programmed address” of the gateway instance. Hostnames defined in the Gateway CR’s listeners will effectively become CNAMEs for that provided A-record.

Required configuration for OpenStack and Private Clouds

In environments like OpenStack, This content is not included.CodeReady Containers, and certain other private cloud deployments, the cluster often lacks native integration for automatically updating or managing external DNS records.

Because the new LoadBalancer Service's external IP address is dynamically provisioned and only becomes known after the RHOAI operator has successfully deployed RHOAI, a manual post-installation step is required to ensure RHOAI components are accessible. If this manual step is skipped, RHOAI components will be inaccessible from outside the cluster.

Post-Installation Requirement: Manual DNS Configuration

Customers running RHOAI 3.x on OpenStack or similar private cloud infrastructure must perform the following manual DNS configuration steps after the RHOAI installation is complete.

Procedure

The following high-level steps are required to configure external DNS:

  1. Retrieve the external LoadBalancer IP
    Identify the external IP address(es) assigned to the new Gateway LoadBalancer Service provisioned by the RHOAI operator.

  2. Access External DNS Provider
    Log in to your external DNS management system (e.g., corporate DNS server, cloud provider DNS service).

  3. Create/Update DNS Record(s)
    Manually create the required A record(s) or CNAME record(s) within your DNS provider. These records must point to the LoadBalancer IP address(es) retrieved in the previous step.

Workaround 1 : For bare-metal node cluster - MetalLB Operator Installation Steps

NOTE : For cloud instances (AWS, Azure, GCP etc.) nodes the default LoadBalancer is used to obtain External IP for the gateway service hence there is no extra step involved. However for bare-metal nodes, you need to setup MetalLB operator instance in the cluster.

  1. Install and create instance for MetalLB Operator
  • Login to Openshift Web UI as kude:admin or user part of cluster-admins.
  • Install MetalLB Operator as per below path.
Openshift Dashboard -> Operators -> Operator Hub -> Search MetalLB -> Install
  • Create instance for type MetalLB.
Openshift Dashboard -> Operators -> Installed Operators -> MetalLB Operator -> MetalLB -> Create MetalLB -> Default Settings -> Create
  1. Assign RHOAI IPAddress Pool
  • Allocate public ipaddress subnet for RHOAI.
  • Create and edit file rhoai_public_ipaddress_pool.yaml
# vi rhoai_public_ipaddress_pool.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: rhoai-public-ips
  namespace: metallb-system
spec:
  addresses:
  - <Subnet>
  autoAssign: true

Note: Replace <Subnet> above with subnet pool (example: 192.168.122.0/24) or individual ipaddress (example: 192.168.122.7/32).

  • Deploy the RHOAI public address pool.
# oc create -f rhoai_public_ipaddress_pool.yaml
  1. Create L2Advertisement for RHAOI.
  • Create and edit file rhoai-l2a.yaml.
# vi rhoai-l2a.yaml
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: primary-network
  namespace: metallb-system
spec:
  ipAddressPools:
  - rhoai-public-ips

Note: The name rhoai-public-ips above should match with name in step 2.

  • Deploy the L2Advertisement.
# oc create -f rhoai-l2a.yaml
  1. Check the service if the RHOAI public ipaddress has been allocated successfully.
$ oc get svc -n openshift-ingress 
$ oc get gateway -n openshift-ingress data-science-gateway -o jsonpath='{.status.addresses[*].value}'

5. Configure the ipaddress in DNS pointed to RHOAI Dashboard URL.

# nslookup <RHOAI Dashboard URL>

6. Try accessing RHOAI Web UI from command line and web browser.

# curl -kv https://<RHOAI Dashboard URL>/

Note: Replace <RHOAI Dashboard URL> with actual fqdn of RHOAI Dashboard URL.

Workaround 2 : Use an OCP Route CR to send traffic to the gateway’s service.

Please note that we are not moving away from Routes completely, however the default behaviour has been changed from Route to Gateway in RHOAI 3.0 version. It is still possible to configure Route that points to your Gateway service to enable external access, with this workaround, you need not always need to use MetalLB as load balancer for bare-metal systems.

In case if you do not want to create MetalLB instance for bare metal cluster, you can create custom resource route instead of load balancer service which will send traffic to gateway service. This Content from github.com is not included.helper script can be used to create route which will send traffic to gateway's service.

Important Considerations

  • Hostname Correspondence: Ensure that the DNS record you create corresponds to the hostname configured for your RHOAI installation (e.g., data-science-gateway.apps.<cluster-url>).
  • A Record vs. CNAME: Depending on your environment and configuration, you will typically create an A record pointing directly to the IP address(es). In some configurations, a CNAME may be used if pointing to a hostname provided by the load balancer service.

Locating the LoadBalancer IP Address

To retrieve the LoadBalancer IP Address, you can use the following oc command:

oc get gateway -n openshift-ingress data-science-gateway -o jsonpath='{.status.addresses[*].value}'

Future Plan

We are working on a slight change to the default architecture that will include an ocp route object out of the box and switch the gateway to “ClusterIP” instead of a loadbalancer for RHOAI Dashboard Service.

  • Future release by default will support Route method just was the case in earlier RHOAI versions.
  • Future release will have a flag in the GatewayConfig CR that allows the admin to switch to “loadbalancer” mode (without an ocp route) if desired.
  • This is being targeted for RHOAI 3.2.
Product(s)
Category
Tags
Article Type