Red Hat OpenShift Service Mesh 3 operator has version higher than what is supported on this cluster version

Solution Verified - Updated

Issue

  • During the incident window (see Red Hat Operator has version higher than the cluster version), some clusters on OCP 4.12–4.17 could receive 4.18 catalog content. Clusters with automatic install plan approval may have upgraded the Service Mesh 3 operator to a version from the 4.18 catalog (e.g. 3.2.1). See the OpenShift Operator Life Cycles for detailed information on which operator versions are supported on each version of OpenShift.
  • OSSM 3.2.z is not supported on OpenShift 4.14, 4.15, 4.16 or 4.17. Those clusters are in an unsupported state and do not receive security updates for that operator version.
  • OSSM 3.1.z is not supported on OpenShift 4.16 or 4.17. Those clusters are in an unsupported state and do not receive security updates for that operator version.

Resolution

To resolve the issue, affected customers must manually downgrade their OSSM operator to a version that is supported on their OpenShift version.

  • On OpenShift 4.14 and 4.15, use the stable-3.0 channel
  • On OpenShift 4.16 and 4.17, use the stable-3.1 channel

Prerequisites:

  • Make sure that you are using an Istio minor version that is supported by the operator version you are targeting to downgrade to.
    • OSSM 3.1.z supports Istio 1.24.z and 1.26.z
    • OSSM 3.0.z supports Istio 1.24.z
  • Also verify that your IstioCNI resource is using a supported version.

Steps:

  1. Backup all your OSSM configurations.

  2. If customers previously used OSSM 3.1.x and had a ZTunnel instance deployed, they will need to remove that instance before downgrading. It can be recreated when the downgrade is complete.

    oc delete ztunnel <ztunnelName>

  3. Delete the operator subscription.

    oc delete subscription servicemeshoperator3 -n openshift-operators

    NOTE: If you are using the OCP Console to remove the subscription, do not check the box that says "delete operands". If you check that box, your mesh will be impacted.

  4. Delete the operator CSV. Note: replace 3.2.1 with your actual installed operator version!

    oc delete csv servicemeshoperator3.v3.2.1 -n openshift-operators

  5. Delete the ZTunnel CRD.

    oc delete crd ztunnels.sailoperator.io

  6. Recreate the operator subscription, e.g. using the OpenShift console's OperatorHub. Ideally, use a channel that pins the minor version (e.g. stable-3.0) and disable automatic install plan approvals.

    • On OpenShift 4.14 and 4.15, use the stable-3.0 channel which should install OSSM 3.0.8 (at time of writing)
    • On OpenShift 4.16 and 4.17, use the stable-3.1 channel which should install OSSM 3.1.5 (at time of writing)

  7. If you previously deleted a ZTunnel instance, recreate it.

Root Cause

Refer to Red Hat Operator has version higher than the cluster version

Diagnostic Steps

To see the version of the OSSM3 operator installed on your system, you can run:

oc get subscription servicemeshoperator3 -n openshift-operators -o jsonpath='{.status.installedCSV}'
  • On OpenShift 4.14 and 4.15, only the OSSM 3.0.z release stream is supported
  • On OpenShift 4.16 and 4.17, both OSSM 3.0.z and OSSM 3.1.z are supported
SBR
Product(s)
Components
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.