Using Openshift Container's service serving certificates in applications
Environment
- Red Hat Openshift Container Platform
- 4.x
- Service serving certificates
Issue
- How to use Openshift Container's service serving certificates in applications running on OpenShift?
- PEM file in OCP service serving certificate?
Resolution
Service CA operator provides features to deploy application in Openshift and inject/use certificate both instrumenting configmap and service (in summary) - these certificates are issued as TLS web server certificates. Below a comparative approach is done for the two types of Service-CA's annotations:
- Service CA serving as service injection
- Service CA serving as configmap injection
The injections are done by the same component Service-CA, but result in different outcomes. There is a third type of injection via Cluster Network labels, see on Root Cause for comparison. Specifically for Java applications, the solution Injecting CA certificate inside OpenJDK container at runtime and build time, would be a specific example of this solution.
Service CA serving as service injection
Create the application and its service, and then annotate with service.beta.openshift.io/serving-cert-secret-name=<secret_name> annotation - the Service CA will then create a configmap with two files: tls.crt and tls.key. In the annotation the user sets the secret that Service CA will inject the data.
The Service CA controller uses the x509.SHA256WithRSA signature algorithm to generate service certificates. And the generated certificate and key are in PEM format, stored in tls.crt and tls.key respectively, within a created secret.
Meaning there will be two files:
tls.crttls.key
Not necessarily one PEM file. That's because the PEM file is a file format/encoding format that may consist of a certificate (also known as the public key) or a private key or both files combined, that's given to the fact that PEM is not a certificate, it is just the encoding format.
For instance, this is used by DataGrid Operator for example in Infinispan Custom Resources.
To use it, set the service.beta.openshift.io/serving-cert-secret-name annotation as below:
oc annotate service <service_name> \1
service.beta.openshift.io/serving-cert-secret-name=<secret_name>
Example:
kind: Secret
apiVersion: v1
metadata:
name: example-infinispan-cert-secret
namespace: encrypt
annotations:
openshift.io/description: 'Secret contains a pair signed serving certificate/key that is generated by Service CA operator for service/example-infinispan with hostname example-infinispan.encrypt.svc and is annotated to the service with annotating a service resource with ''service.beta.openshift.io/serving-cert-secret-name: example-infinispan-cert-secret''. The certificate is valid for 2 years.'
openshift.io/owning-component: service-ca
service.alpha.openshift.io/expiry: '2027-05-13T22:10:23Z'
service.beta.openshift.io/expiry: '2027-05-13T22:10:23Z'
service.beta.openshift.io/originating-service-name: example-infinispan
service.beta.openshift.io/originating-service-uid: 089a9f20-db6b-47fe-a678-d1240c619aad
ownerReferences:
- apiVersion: v1
kind: Service
name: example-infinispan
data:
tls.crt:...
tls.key:...
type: kubernetes.io/tls
Service CA serving as configmap injection
Also, it is possible for a pod to access the service CA certificate by mounting a ConfigMap object that is annotated with service.beta.openshift.io/inject-cabundle=true.
For that create a configmap and annotate with service.beta.openshift.io/inject-cabundle=true, this will make the Service-CA create a certificate service-ca.crt that can be referenced in the deployment, as below:'
volumeMounts:
- mountPath: /etc/pki/ca-trust/source/anchors/ <--- not /etc/pki/ca-trust/extracted/pem.
name: trusted-ca
readOnly: true
...
volumes:
- configMap:
defaultMode: 420
items:
- key: service-ca.crt <------------------ verify the configmap has this data
path: example.pem
name: ca-inject
name: trusted-ca
Location: /etc/pki/ca-trust/source/anchors/ should be used rather than /etc/pki/ca-trust/extracted/pem, because nothing should not be directly overwriting extracted/*.
Configmap:
apiVersion: v1
data:
service-ca.crt: |
-----BEGIN CERTIFICATE-----
MIIDUTC...wH+KMndrHpg=
-----END CERTIFICATE-----
kind: ConfigMap
Root Cause
As explained in OCP 4.18 Security and compliance - Chapter 3. Configuring certificates, the Service CA operator provides features to deploy application in Openshift and inject/use certificates, the Service CA will be located in the openshift-service-ca and openshift-service-ca-operator. It is possible to track configmap creation by the service-ca Operator operand logs, see Diagnostic Steps.
Service-CA Operator can provide certificates in two approaches:
| Service CA Method | Explanation | Details |
|---|---|---|
When the user creates a configmap or webhook and annotates with service.beta.openshift.io/inject-cabundle=true | Service-CA Operator will inject a service-ca.crt key into the configmap - to be set on the deployment yaml as volume mount/path | Verify the configmap spec.data.service-ca.crt |
When the user sets a service with the annotation service.beta.openshift.io/serving-cert-secret-name by giving the name of the service's serving secret to be created | Service-CA Operator will create a secret certificate with data: tls.crt and tls.key | Verify the secret created: it will have a pair signed serving certificate/key |
Also, note that service.beta.openshift.io/inject-cabundle=true will collide with the Cluster Network's label: config.openshift.io/inject-trusted-cabundle, which injects a ca-bundle as well as clarified on the Cluster Network.
The Service-CA certificate is used for internal communication handling (with annotation service.beta.openshift.io/inject-cabundle: true). And contrasts with the global environment (node) certificate CA's for external calls which is done via the label: config.openshift.io/inject-trusted-cabundle=true - see details below.
In summary:
| Component | Usage | Purpose | Result | Certificate details |
|---|---|---|---|---|
| Cluster Network | add label: config.openshift.io/inject-trusted-cabundle | Its purpose is to use for Operators, not necessarily custom applications | Result in ca-bundle.crt configmap with CNO as owning-component annotation | ca-bundle.crt includes several certificates includes External CA certificates (several types) |
| Service CA Operator | Add service.beta.openshift.io/inject-cabundle=true or service.beta.openshift.io/serving-cert-secret-name | The Service CA Operator will provide the certificate injection as details on this solution | Result in service-ca.crt configmap with Service-CA as owning-component annotation. | service-ca.crt includes one certificate (no specific singing detailed) from openshift-service-ca's configmap signing-cabundle |
Details on the certificate rotation
The certificate is created/controled by the Service-CA Operator. Regarding automatic triggering a renew of the certificate, there is no specific trigger for automatic renew, nor the respective Custom Resources allow this features. Therefore, to force a trigger, delete the service instrumented, this will force a new certificate to be created.
Details on the certificate injected
Service-CA Certificate Injection
The certificate is not created each time necessarily, it just contains the PEM-encoded CA signing bundle from the openshift-service-ca namespace:
$ oc project java-injection
oc get cm
ca-inject 1 62m
...
...
$ oc get cm ca-inject -o yaml
apiVersion: v1
data:
service-ca.crt: |
-----BEGIN CERTIFICATE-----
MIIDUTCCA....+KMndrHpg==
-----END CERTIFICATE-----
kind: ConfigMap
metadata:
annotations:
...
openshift.io/owning-component: service-ca
service.beta.openshift.io/inject-cabundle: "true"
This comes from the certificate locate in the openshift-service-ca namespace:
$ oc project openshift-service-ca
Now using project "openshift-service-ca" on server ...
$ oc get cm
NAME DATA AGE
kube-root-ca.crt 1 5d3h
openshift-service-ca.crt 1 5d3h
signing-cabundle 1 5d3h
$ oc get cm signing-cabundle -o yaml
apiVersion: v1
data:
ca-bundle.crt: |
-----BEGIN CERTIFICATE-----
MII...KsHg==
-----END CERTIFICATE-----
kind: ConfigMap
metadata:
annotations:
openshift.io/description: Service CA configmap contains the data for the PEM-encoded
CA signing bundle which will be injected to resources annotated with 'service.beta.openshift.io/inject-cabundle=true'
openshift.io/owning-component: service-ca
name: signing-cabundle
namespace: openshift-service-ca
CNO's Certificate Injection
Using the label config.openshift.io/inject-trusted-cabundle: 'true', the CNO (together with the Service-CA Operator) will inject several Root Certificates from different authorities:
labels:
config.openshift.io/inject-trusted-cabundle: 'true'
annotations:
openshift.io/owning-component: Networking / cluster-network-operator
...
data:
ca-bundle.crt: |
# ACCVRAIZ1
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE------
...
AC RAIZ FNMT-RCM SERVIDORES SEGUROS
ANF Secure Server Root CA
Actalis Authentication Root CA
AffirmTrust Commercial
AffirmTrust Networking
AffirmTrust Premium
...
GlobalSign Root CA - R3
GlobalSign Root CA - R6
Diagnostic Steps
- Verify the files created on namespace:
openshift-service-ca:
$ oc logs service-ca-id
I0513 19:14:40.421868 1 configmap.go:109] updating configmap data-grid-ns/openshift-service-ca.crt with the service signing CA bundle
I0513 22:09:52.785177 1 configmap.go:109] updating configmap encrypt/openshift-service-ca.crt with the service signing CA bundle
- From that pod logs, it is possible to track the specific configmap was created - e.g.
openshift-service-ca.crt:
$ oc get cm openshift-service-ca.crt -o yaml
apiVersion: v1
data:
service-ca.crt: | -----------------------------------> certificate(crt) with X509 standard content format
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
kind: ConfigMap
metadata:
annotations:
service.beta.openshift.io/inject-cabundle: "true"
- Example below is a secret generated by the service-ca for the service:
example-infinispan, as shown byopenshift.io/owning-component: service-ca:
$ oc get secret example-infinispan-cert-secret -o yaml
apiVersion: v1
data:
tls.crt: (...)
tls.key: (...)
kind: Secret
metadata:
annotations:
openshift.io/description: 'Secret contains a pair signed serving certificate/key
that is generated by Service CA operator for service/example-infinispan with
hostname example-infinispan.encrypt.svc and is annotated to the service with
annotating a service resource with ''service.beta.openshift.io/serving-cert-secret-name:
example-infinispan-cert-secret''. The certificate is valid for 2 years.'
openshift.io/owning-component: service-ca
service.alpha.openshift.io/expiry:
service.beta.openshift.io/expiry:
service.beta.openshift.io/originating-service-name: example-infinispan
service.beta.openshift.io/originating-service-uid: (...)
name: example-infinispan-cert-secret
namespace: encrypt
The service must be created with the annotation service.beta.openshift.io/serving-cert-secret-name as below:
oc get svc example-infinispan -o yaml
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.openshift.io/serving-cert-secret-name: example-infinispan-cert-secret
Forcing re-spin:
$ oc extract secret/example-infinispan-cert-secret --keys=tls.crt --to=- | openssl x509 -noout -dates
# tls.crt
notBefore=Dec 17 18:17:12 2025 GMT
notAfter=Dec 17 18:17:13 2027 GMT
$ oc delete secret/example-infinispan-cert-secret
secret "example-infinispan-cert-secret" deleted
$ oc extract secret/example-infinispan-cert-secret --keys=tls.crt --to=- | openssl x509 -noout -dates
# tls.crt
notBefore=Dec 18 18:49:12 2025 GMT
notAfter=Dec 18 18:49:13 2027 GM
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.