How to create a file-based catalog format in Openshift 4.10 and newer

Solution Unverified - Updated 13 Jun 2024

Environment

Red Hat Openshift Container Platform (RHOCP) 4.10

Issue

How to filter and pull the redhat-operator-index to a connected mirror registry
How to export the filtered redhat-operator-index to a file structure for transfer
How to import the file based redhat-operator-index into a disconnected mirror registry

Resolution

The following process assumes that both the connected and disconnected mirror registries are already deployed. An example of such a registry can be found at Creating a mirror registry with mirror registry for Red Hat OpenShift

Pre-requesit installs
- Operator Package Manager CLI (Installing the opm CLI)
- jq
- podman
Authenticate with the Red Hat registry using podman
```
$ podman login registry.redhat.io
```

Download the full operator index using opm

$ opm render registry.redhat.io/redhat/redhat-operator-index:v4.10 > index.json
WARN[0094] DEPRECATION NOTICE:
Sqlite-based catalogs and their related subcommands are deprecated. Support for
them will be removed in a future release. Please migrate your catalog workflows
to the new file-based catalog format.

Create folder structure to work from

$ mkdir -p catalog/redhat-operator-index; cd catalog

Generate a Containerfile based upon the redhat-operator-index

    $ cat << EOF > redhat-operator-index.Containerfile
    # The base image is expected to contain /bin/opm
    # (with a serve subcommand) and /bin/grpc_health_probe
    FROM registry.redhat.io/openshift4/ose-operator-registry:v4.10

    # Configure the entrypoint and command
    ENTRYPOINT ["/bin/opm"]
    CMD ["serve", "/configs"]

    # Copy declarative config root into image at /configs
    ADD redhat-operator-index /configs

    # Set DC-specific label for the location of the DC root directory
    # in the image
    LABEL operators.operatorframework.io.index.configs.v1=/configs
    EOF

To determine the names of the packages contained in the index use jq to list the packages

    $ jq .package ../index.json | jq -s 'unique_by(.)'

    [
      ...
      "ansible-automation-platform-operator",
      "ansible-cloud-addons-operator",,
      ...
      "ptp-operator",
      "quay-bridge-operator",
      "quay-operator",
      "red-hat-camel-k",
      "redhat-oadp-operator",
      "rh-service-binding-operator",
      ...
    ]

Select the operators required by using jq to capture the data and write it to a new index file. In this example the quay-operator has been selected

$ jq '. | select((.package=="quay-operator") or (.name=="quay-operator"))' ../index.json > redhat-operator- index/index.json
$ jq '. | select((.package=="ansible-automation-platform-operator") or (.name=="ansible-automation-platform-operator"))' ../index.json >> redhat-operator- index/index.json

Use opm to validate the index.json
```
$ opm validate redhat-operator-index
```

Build and tag the image containing the filtered operators

$ podman build -f redhat-operator-index.Containerfile -t mirror.ocp.example.local:5000/olm/redhat-operator-index:v4.10
STEP 1/5: FROM registry.redhat.io/openshift4/ose-operator-registry:v4.10
STEP 2/5: ENTRYPOINT ["/bin/opm"]
--> Using cache 24157b2ef98175787a8f8a2c3a4bad3f04b73ce87d527d44ba61f3f1e85a9530
--> 24157b2ef98
STEP 3/5: CMD ["serve", "/configs"]
--> Using cache 42ba5e5af8dc0f417ed5d65b6362afbd7040e96c0b36b5fa55c0a5c20d93b813
--> 42ba5e5af8d
STEP 4/5: ADD redhat-operator-index /configs
--> bc6abaf55c1
STEP 5/5: LABEL operators.operatorframework.io.index.configs.v1=/configs
COMMIT localhost:8443/olm/redhat-operator-index:v4.10
--> 73a3ce5fff6
Successfully tagged mirror.ocp.example.local:5000/olm/redhat-operator-index:v4.10
73a3ce5fff6cd45edef57034b11eac7965682a805a4e0a3769bf5153e6552c9c

Authenticate with the local mirror registry

$ podman login -u <username> -p <password> mirror.ocp.example.local:5000 --tls-verify=false

Push the operators image into the local mirror registry

$ podman push mirror.ocp.example.local:5000/olm/redhat-operator-index:v4.10 --tls-verify=false
Copying blob 6ef5f78e84f4 done  
Copying blob d68a9c66ac3c done  
Copying blob 7c384a39fa5a done  
Copying blob 03882987b785 done  
Copying blob e986b6483de0 done
Copying blob 11088524b464 done  
Copying config 73a3ce5fff done  
Writing manifest to image destination
Storing signatures

Create an authfile containing pull secrets and add the credentials for the local registry instance to it by following this article Configuring credentials that allow images to be mirrored

Run the catalog mirror command to export the operators to file

$ oc adm catalog mirror -a /path-to-secrets-file/pull-secret.json mirror.ocp.example.local:5000/olm/redhat-operator-index:v4.10 mirror.ocp.example.local:5000/olm --to-manifests=$(pwd)/manifests --insecure --continue-on-error=true
rc image has index label for declarative configs path: /configs/
using index path mapping: /configs/:/tmp/1734716778
wrote declarative configs to /tmp/1734716778
using declarative configs at: /tmp/173471677
uploading: mirror.ocp.example.local:5000/olm/quay-quay-operator-bundle 
sha256:8e905298ff645ec844076b2c175f8a3094bdb9ad5ef5c5a9cdde790ffecd8e5b 44.29KiB
uploading: mirror.ocp.example.local:5000/olm/quay-quay-operator-bundle 
sha256:fd79d38760a060a9adf56d8212ca3b4ac69ec6209fd7ad1c75dd6879a5d29fef 40.56KiB
mounted: mirror.ocp.example.local:5000/olm/olm1-redhat-operator-index 
sha256:9740c78f7202ef664d1c2b51f9d9b03131113633025d879fc61391e9ab0253e7 133MiB
mounted: mirror.ocp.example.local:5000/olm/olm1-redhat-operator-index 
sha256:2d82beaa9f7a5f6202ce07b12f0d9218f181ec2529a186a1e6979890edb590aa 1.269MiB
...
...
info: Mirroring completed in 6m23.83s (7.553MB/s)
wrote mirroring manifests to ../catalog/manifests

Create an folder structure for the export files
```
$ mkdir ../olm_files; cd ../olm_files
```

Export the filtered redhat-operator-index to file for transfer

    $ oc adm catalog mirror mirror.ocp.example.local:5000/olm/redhat-operator-index:v4.10 file:///redhat-operator-index --insecure --continue-on-error=true

    ...
    47ae3e5dc7085028eb259e3444d50c341838 87.07MiB
          registry.redhat.io/rhel8/postgresql-10 sha256:ac98a42112890b4e2908f55aa520de5b0228da1e03ef3f762bd7e20ba7fe4ec9 87.1MiB
        manifests:
          sha256:05f69e06541e1b01fc7b7b93a3ee46c314c2eec853703706b30b9f9a7df1c289
          sha256:09bc56a6b1bddc29a41e616ecff294142fedf3256e5955b1df259d07488bab26
          sha256:0b17d6def11c9fd6d1c1230ff390b91a22ff866a738bb2afe1bca644e5e256d0
    ...
    info: Mirroring completed in 5m41.35s (10.62MB/s)
    wrote mirroring manifests to manifests-redhat-operator-index-1670606531

    To upload local images to a registry, run:

	   oc adm catalog mirror file://redhat-operator-index/olm1/redhat-operator-index:v4.10 REGISTRY/REPOSITORY
    deleted dir /tmp/2527342782

Check the files have been created and the structure is correct running a tree command

    ...olm_files]$ tree -L 6
    .
    ├── manifests-redhat-operator-index-1670606501
    ├── manifests-redhat-operator-index-1670606531
    │   └── mapping.txt
    └── v2
        └── redhat-operator-index
            └── olm
                └── redhat-operator-index
                    ├── blobs
                    │   ├── sha256:2d82beaa9f7a5f6202ce07b12f0d9218f181ec2529a186a1e6979890edb590aa
                    │   ├── sha256:712e22c200ed951410c19c9f3e760446e1e516c5bb9dcb6d53033c0dbfef18d5
                    │   ├── sha256:73a3ce5fff6cd45edef57034b11eac7965682a805a4e0a3769bf5153e6552c9c
                    │   ├── sha256:7cde10acfa2e4561b4be25735bb4b9be264f0f4d259bbe275b97b5e7f8839b5c
                    │   ├── sha256:9740c78f7202ef664d1c2b51f9d9b03131113633025d879fc61391e9ab0253e7
                    │   ├── sha256:9df9aa2998cab73cb8c78646332865ce19dd4dc59f2d0d246b82d52befa48d22
                    │   ├── sha256:f0e75f0712cb54427ddf23f001cb109d505557b677f932717bbd1e6219bcecb3
                    │   └── sha256:f2c1a2360be4889ba8b66153bd01031a18129c7a932a863da4913751dc93d1fa
                    ├── manifests
                    │   ├── sha256:712e22c200ed951410c19c9f3e760446e1e516c5bb9dcb6d53033c0dbfef18d5
                    │   └── v4.10 -> sha256:712e22c200ed951410c19c9f3e760446e1e516c5bb9dcb6d53033c0dbfef18d5
                    ├── quay
                    │   ├── clair-rhel8
                    │   ├── quay-builder-qemu-rhcos-rhel8
                    │   ├── quay-builder-rhel8
                    │   ├── quay-operator-bundle
                    │   ├── quay-operator-rhel8
                    │   ├── quay-rhel8
                    │   └── quay-rhel8-operator
                    └── rhel8
                        ├── postgresql-10
                        ├── redis-5
                        └── redis-6

The data can now be taken to the disconnected cluster and imported

Root Cause

Organizational control determines that operators are limited in the disconnected cluster
Space is limited in the disconnected cluster and only a subset of a catalog is required

SBR

Shift

Product(s)

Red Hat OpenShift Container Platform

Category

Configure

Tags

openshift

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.