OpenShift builds fail trying to push image using a wrong IP address for the registry

Environment

• OpenShift Enterprise 3.0

Issue

• How can I specify a different IP address for the internal registry on OpenShift v3?
• In my OpenShift v3 environment, the integrated docker registry service IP changed since its original deployment as a consequence of a change in the infrastructure. When I create new builds they all fail because they are trying to push the generated images using the old IP address for the registry.
• STI Build Pushing to Wrong Registry IP

Resolution

NOTE: This is a solution to override the value of the registry service's IP. If you would like to maintain the registry IP, please refer to How to retain or manually set the service IP address for docker-registry in Openshift 3 .

Restart the OpenShift master service:

# systemctl restart atomic-openshift-master

If this does not resolve the problem, or if you want to override the value of the registry service's IP, you can manually add this to /etc/sysconfig/atomic-openshift-master:

OPENSHIFT_DEFAULT_REGISTRY=172.30.133.45:5000

And restart your master service with systemctl restart atomic-openshift-master.

Root Cause

When you recreate the service associated to the internal registry it will receive a new IP address. At the time of this writing, the OpenShift master does not automatically detect that change and needs to be updated manually via a service restart.

Diagnostic Steps

You can see which address is going to be used to push images from builds by looking at the ImageStreams associated with them.

For example, for this buildConfig:

$ oc get bc nodejs-example -o yaml
apiVersion: v1
kind: BuildConfig
metadata:
  annotations:
# Please edit the object below. Lines beginning with a '#' will be ignored,
    description: Defines how to build the application
  creationTimestamp: 2015-07-10T11:31:42Z
  labels:
    template: nodejs-example
  name: nodejs-example
  namespace: myproject
  resourceVersion: "97558"
# Please edit the object below. Lines beginning with a '#' will be ignored,
  selfLink: /osapi/v1beta3/namespaces/myproject/buildconfigs/nodejs-example
  uid: 3ceedf30-26f7-11e5-ac35-fa163e4fb35b
spec:
  output:
    to:
# Please edit the object below. Lines beginning with a '#' will be ignored,
      kind: ImageStreamTag
      name: nodejs-example:latest
  resources: {}
  source:
    git:
      uri: https://github.com/openshift/nodejs-ex.git
    type: Git
  strategy:
    sourceStrategy:
      from:
        kind: ImageStreamTag
        name: nodejs:0.10
        namespace: openshift
    type: Source
  triggers:
  - imageChange:
      lastTriggeredImageID: registry.access.redhat.com/openshift3/nodejs-010-rhel7:latest
    type: ImageChange
  - github:
      secret: UOks8OFmsJVQDEEhM5iOj2mEc1v4gLidd6ad3BH2
    type: GitHub
status:
  lastVersion: 2

you can see that the spec section says that the output of the builds should go to nodejs-example:latest.

You will see where the images are going to be pushed by exploring the associated ImageStream:

$ oc get is
NAME             DOCKER REPO                             TAGS      UPDATED
nodejs-example   172.30.220.16:5000/myproject/nodejs-example

$ oc get is nodejs-example -o yaml
apiVersion: v1
kind: ImageStream
metadata:
  annotations:
    description: Keeps track of changes in the application image
  creationTimestamp: 2015-07-10T11:31:42Z
  labels:
    template: nodejs-example
  name: nodejs-example
  namespace: myproject
  resourceVersion: "97397"
  selfLink: /osapi/v1beta3/namespaces/myproject/imagestreams/nodejs-example
  uid: 3cdd1c99-26f7-11e5-ac35-fa163e4fb35b
spec: {}
status:
  dockerImageRepository: 172.30.220.16:5000/myproject/nodejs-example

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.