oc-mirror in a disconnected environment fails with a "HTTP 401 Unauthorized" error

Environment

• Red Hat OpenShift Container Platform (RHOCP)
  • 4.9+
• oc-mirror
  • v1
  • v2

Issue

• "oc-mirror" retrieves registry credentials from ~/.docker/config.json, that is inconvenient that a user is forced to add credentials to such a generic location for mirroring purposes. How can this path be defined in another way?

• When trying to run the "oc-mirror" plugin without having a matching registry credential in one of ${HOME}/.docker/config.json or ${XDG_RUNTIME_DIR}/containers/auth.json paths against a private Container Image Registry a 401 Unauthorized error is returned:

    $ ./oc-mirror --config imageset-config.yaml file://archive
    Creating directory: archive/oc-mirror-workspace/src/publish
    Creating directory: archive/oc-mirror-workspace/src/v2
    Creating directory: archive/oc-mirror-workspace/src/charts
    Creating directory: archive/oc-mirror-workspace/src/release-signatures
    backend is not configured in imageset-config.yaml, using stateless mode
    backend is not configured in imageset-config.yaml, using stateless mode
    No metadata detected, creating new workspace
    level=info msg=trying next host error=failed to authorize: failed to fetch anonymous token: unexpected status: 401 Unauthorized host=registry.redhat.ioThe rendered catalog is invalid.Run "oc-mirror list operators --catalog CATALOG-NAME --package PACKAGE-NAME" for more information.error: error rendering new refs: render reference "registry.redhat.io/<any-namespace>/<image-name>:<image-tag>": error resolving name : failed to authorize: failed to fetch anonymous token: unexpected status: 401 Unauthorized
  the command terminates with HTTP401 Unauthorized which is something that we expect since there is no `config.json` file created.
  

Resolution

• You can use any config.json file available at any path (eventually more secure) by exporting the DOCKER_CONFIG environment variable, i.e.

  $ export DOCKER_CONFIG=/home/user/my-secure-path
  $ ls -l /home/user/my-secure-path/config.json
  -rw-------. 1 user user 2915 Jan  1  1970 /home/user/my-secure-path/config.json
  $ ls -lZ /home/user/my-secure-path/config.json
  -rw-------. 1 user user unconfined_u:object_r:user_home_t:s0 163 Jan  1  1970 /home/user/my-secure-path/config.json
  $ jq -S . /home/user/my-secure-path/config.json
  {
    "auths": {
      "registry.redhat.io": {
        "auth": "<base64-encoded-string>"
      }
    }
  }
  

• PLEASE NOTE: File mode should be 0600

• And perform the container base image mirroring with oc-mirror

  $ ./oc-mirror --config imageset-config.yaml file://archive
  Found: archive/oc-mirror-workspace/src/publish
  Found: archive/oc-mirror-workspace/src/v2
  Found: archive/oc-mirror-workspace/src/charts
  Found: archive/oc-mirror-workspace/src/release-signatures
  backend is not configured in imageset-config.yaml, using stateless mode
  backend is not configured in imageset-config.yaml, using stateless mode
  No metadata detected, creating new workspace
  
  [OUTPUT OMITTED FOR SHORTING PURPOSES]
  
  info: Mirroring completed in xx.yys (xxx.yyyMB/s)
  Creating archive /home/user/ocp-cluster/example-image/archive/mirror_seq1_000000.tar
  

• In this example we see registry.redhat.io which is an authenticated Container Image Registry (see Red Hat Container Registry Authentication)

Root Cause

Disclaimer: Links contained herein to external website(s) are provided for convenience only. Red Hat has not reviewed the links and is not responsible for the content or its availability. The inclusion of any link to an external website does not imply endorsement by Red Hat of the website or its entities, products, or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) the external site or content.

• Being oc-mirror a Golang written tool it is capable of inheriting environment variables from the underlying shell being used (see Content from pkg.go.dev is not included.Go Docs - Environment Variables). So, by design, the plugin retrieves registry credentials from the followings paths:

  • ${HOME}/.docker/config.json
  • ${XDG_RUNTIME_DIR}/containers/auth.json

• And any paths exported by the environment variable:

  • DOCKER_CONFIG

Diagnostic Steps

• Trying to mirror an image from a private Container Image Registry over a disconnected RHOCP environment fails with the error HTTP401 Unauthorized using the oc-mirror tool

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.