Long-lived service account API tokens in OpenShift Container Platform
Kubernetes API tokens were originally non-expiring and persisted in secrets. These long-lived tokens presented some risks to information security because they do not expire and some scalability issues because each token requires a secret to persist it. In an OpenShift Container Platform cluster, access to the Secrets containing these API tokens are typically protected using the cluster’s user authentication and authorization system.
To enhance the security of API tokens, Kubernetes introduced the ability to create API tokens with bounded lifetimes. These bound API tokens are invalidated after a set expiration time and can additionally be bound to specific resources, which will invalidate the API token when the resource is deleted.
Long-lived API tokens, also referred to as “legacy” tokens now, can still be created if needed. This article enumerates the legacy API tokens that might still be found in an OpenShift cluster.
API tokens generated by the Kubernetes controller manager
Before OpenShift Container Platform v4.11, the bundled version of Kubernetes automatically generated a legacy service account API token secret for every service account in the cluster. In OpenShift Container Platform v4.11, the LegacyServiceAccountTokenNoAutoGeneration Kubernetes feature was enabled, which disables the automatic generation of these service account API tokens. As of OpenShift Container Platform v4.15, the LegacyServiceAccountTokenNoAutoGeneration Kubernetes feature can no longer be disabled1.
API tokens generated by the OpenShift controller manager
In order to integrate the OpenShift Container Platform’s image registry into the cluster’s user authentication and authorization system, a legacy service account API token secret is automatically generated for each service account in the cluster.
Starting with OpenShift Container Platform v4.15, if the ImageRegistry capability is not enabled, or the integrated image registry is disabled, these service account API tokens are not automatically generated.
API tokens generated for control plane node recovery
On an OpenShift Container Platform cluster with a standalone control plane2, when a control plane node is shut down, there is a risk that upon starting it back up, the credentials needed to rejoin the control plane components to the cluster will have expired. Starting in OpenShift Container Platform v4.5 there is an automatic recovery mechanism that detects this situation, recovers the control plane node, and reconnects it to the cluster.
For this mechanism to work, core control plane components are bootstrapped with credentials that allow them to request permission to bootstrap the control plane node and reconnect to the cluster.
| Namespace | Service Account | Secret |
|---|---|---|
| openshift-kube-apiserver | localhost-recovery-client | localhost-recovery-client-token |
| openshift-kube-controller-manager | localhost-recovery-client | localhost-recovery-client-token |
| openshift-kube-scheduler | localhost-recovery-client | localhost-recovery-client-token |
The long-lived API tokens are generated and installed onto the control plane nodes.
API tokens generated for worker node bootstrapping
On an OpenShift Container Platform cluster with a standalone control plane2, when a worker node is shut down, there is a risk upon restarting it that the kubelet’s credentials have expired and it cannot reconnect the node to the cluster. A long-lived API token is created and saved on the node that allows the node to request new credentials from the API server and reconnect the worker node to the cluster.
| Namespace | Service Account | Secret |
|---|---|---|
| openshift-machine-config-operator | node-bootstrapper | node-bootstrapper-token |
The long-lived API tokens are generated and installed onto the worker nodes.
Detecting active usage of legacy service account API tokens
Secrets with the kubernetes.io/legacy-token-last-used label
To list all legacy service account API token secrets in a cluster, filter the list of secrets by the type field. For example:
oc get secrets \
--all-namespaces \
--field-selector type=kubernetes.io/service-account-token
Starting with OpenShift Container Platform v4.14, legacy service account API token secrets will be labeled with the date when the service account API token it contains was last used. When listing legacy service account API token secrets, filter using the kubernetes.io/legacy-token-last-used3 label to get a list of service account API token secrets containing a service account API token that has been used. For example:
oc get secrets \
--all-namespaces \
--show-labels \
--field-selector type=kubernetes.io/service-account-token \
--selector kubernetes.io/legacy-token-last-used
Audit log events with the authentication.k8s.io/legacy-token annotation
Starting with OpenShift Container Platform v4.6, audit log events will contain the authentication.k8s.io/legacy-token annotation if the request presented a legacy service account API token for authentication. The value of the annotation will be the name of the service account.
The entries in the audit log can assist in identifying workloads utilizing legacy service account API tokens. For example, you can identify legacy service account API tokens used for authentication with the integrated image registry as the audit log event will typically have a value beginning with dockerregistry in the userAgent field.
The LegacyServiceAccountTokenNoAutoGeneration feature gate GA'ed in Kubernetes 1.26 where it was defaulted to true (enabled). The feature gate was Content from kubernetes.io is not included.removed in Kubernetes 1.28, removing the ability to disable the feature.
A standalone control plane is hosted directly on dedicated physical or virtual machines. This is in contrast to a This page is not included, but the link has been rewritten to point to the nearest parent document.hosted control plane.
Provided by the upstream LegacyServiceAccountTokenTracking feature, enabled by default Content from kubernetes.io is not included.starting with Kubernetes 1.27.