RHOAI Single-Model Serving Endpoints Return HTTP 403 Errors After Authorino Update

Solution Unverified - Updated

Environment

  • Red Hat OpenShift AI (RHOAI)
    • 2.16
  • Authorino operator tech-preview channel
    • v1.1.1

Issue

  • After Authorino operator upgrade(e.g. tech-preview channel from v1.0.2 to v1.1.1), Red Hat OpenShift AI (RHOAI) Single-Model Serving endpoints start returning HTTP 403 errors

Resolution

Check Authorino Pod Status:

Run:

oc get pods -n redhat-ods-applications-auth-provider

A pod similar to the following indicates no sidecar injection if it shows 1/1 instead of 2/2 for a mesh-integrated Authorino:

NAME                         READY   STATUS    RESTARTS   AGE
authorino-79dd5d6cb7-xs85c   1/1     Running   0          15s

Resolution (Workaround):

  1. After upgrading Authorino, restart RHOAI operator pod:
oc delete pod -l name=rhods-operator -n redhat-ods-operator
  1. Wait for the Authorino pod to restart. After the restart, the pod should show 2/2 containers ready:
oc get pods -n redhat-ods-applications-auth-provider

Example of expected output:

NAME                         READY   STATUS    RESTARTS   AGE
authorino-XXXXX-YYYYY        2/2     Running   0          <time>
  1. Test the model endpoint again

Root Cause

After updating to certain Authorino versions, the automatic Istio sidecar injection may not be reapplied. Without the sidecar, Authorino is not correctly integrated into the service mesh, causing upstream requests to fail with HTTP 403 responses.

Restart the RHOAI operator pod will do a reconciliation of the DSCInitialization resource, which will re-apply the sidecar to the Authorino.

SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.