3scale Operator inadvertently upgraded to version 2.16 on OCP 4.12-4.13 clusters

Environment

• Red Hat OpenShift Container Platform (RHOCP)
  • 4
• Red Hat 3scale API Management On-premises
  • 2.16.0
  • 2.16.1
  • 2.16.2

Issue

After a catalog update, the 3scale Operator may have been upgraded to version v2.16, which is not supported on OpenShift Container Platform (OCP) versions 4.13 or older clusters.

This article covers the specific remediation for the 3scale Operator. For the general issue affecting multiple operators, refer to Red Hat Operator has version higher than the cluster version.

Resolution

Verify 3scale version

oc get APIManager -o json | jq -r ".items[].metadata.annotations"

Take note of "apps.3scale.net/threescale-operator-version" and "apps.3scale.net/apimanager-threescale-version"

Follow Scenario 1 if the value of the "apps.3scale.net/apimanager-threescale-version" is 2.15.x or lower. Otherwise, follow Scenario 2.

Scenario 1: 3scale version remains with the original version, despite the 3scale operator being upgraded to v2.16.

Follow the below steps to recover the 3scale Operator:

• Take a This content is not included.backup of the current 3scale environment.
• Remove the current 3scale 2.16 operator:
  • Navigate to Operators > Installed Operators and select Uninstall Operator option from the three dots menu on the right of the Red Hat Integration - 3scale operator row.
  • You may safely remove or scale down the Operator. While the Operator is running, it reconciles and performs changes within 3scale. If it is scaled down or uninstalled, the Operator will be unable to make any further changes. Crucially, DO NOT remove the APIManager CR or any other associated object. Removing the APIManager CR will result in the complete removal of your 3scale instance.
• Reinstall the 3scale Operator. From the Administrator view, navigating to Operators > OperatorHub, search for 3scale and select Red Hat Integration - 3scale.
  • Click the Install button.
  • Make sure the Update channel match "apps.3scale.net/threescale-operator-version" in previous step
  • Select a specific namespace on the cluster, select the namespace under Installed Namespace and click the Install button.

Senario 2: 3scale installation has been upgraded to v2.16

3scale does not support downgrading. To remain supported, you must upgrade cluster to OCP 4.14 or a newer version. Refer to the supported configuration for more details.

Diagnostic Steps

• Check to see if the incorrect CSV is installed

$ oc get csv -A | grep "Red Hat Integration - 3scale"

3scale-test                            3scale-operator.v0.13.2   Red Hat Integration - 3scale   0.13.2                      Succeeded

In this example from an OpenShift version 4.12 cluster, the CSV version is expected to be version v0.12.x or lower, but instead it is v0.13.x. In this case, the above remediation steps should be followed to downgrade 3scale operator.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.