ElasticSearch unable to start with error "cannot create tokenreviews.authentication.k8s.io at the cluster scope"

Solution Verified - Updated

Environment

Openshift Container Platform 3.7 and above

Issue

The ElasticSearch pod fail to start because of the following error:

unable to load OpenShift configuration: unable to retrieve authentication information for tokens: User "system:serviceaccount:logging:aggregated-logging-elasticsearch" cannot create tokenreviews.authentication.k8s.io at the cluster scope: User "system:serviceaccount:logging:aggregated-logging-elasticsearch" cannot create tokenreviews.authentication.k8s.io at the cluster scope (post tokenreviews.authentication.k8s.io)

Resolution

Run the following command:

$ oc adm policy add-cluster-role-to-user system:auth-delegator -z ${USER} -n ${NAMESPACE}

Where:
${USER} is the user being unable to create tokenreviews, in this case system:serviceaccount:logging:aggregated-logging-elasticsearch
${NAMESPACE} is the namespace where the elasticsearch pod is deployed

Root Cause

The ElasticSearch pod is comprised by two containers: elasticsearch and an oauth-proxy.

The oauth-proxy needs permissions to create tokenreviews and by doing that, authenticate users. If this permission is not granted it will end in errors like this.

Diagnostic Steps

Check the failed ElasticSearch deployment by doing:

$ oc describe ${ES-POD-NAME}

And look for the logs showing permissions errors:

User "system:serviceaccount:logging:aggregated-logging-elasticsearch" cannot create tokenreviews.authentication.k8s.io at the cluster scope: User "system:serviceaccount:logging:aggregated-logging-elasticsearch" cannot create tokenreviews.authentication.k8s.io at the cluster scope (post tokenreviews.authentication.k8s.io)
SBR
Product(s)
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.