External access to Elasticsearch

Solution Verified - Updated

Environment

  • OpenShift Container Platform
    • 3.5
    • 3.6
    • 3.7 and later

Issue

  • Connecting external applications such as Grafana to Elasticsearch
  • Logging in to Elasticsearch from outside of the cluster
  • Curling Elasticsearch externally

Resolution

Connecting to Elasticsearch Externally in OCP 3.7+


Starting in 3.7, this is a [fully documented feature](https://docs.openshift.com/container-platform/3.11/install_config/aggregate_logging.html#expose-elasticsearch-as-route).

Connecting to Elasticsearch Externally in OCP 3.6

The ability to connect to Elasticsearch externally has been added to OpenShift Container Platform 3.6 but is undocumented in 3.6. The same principles in the 3.7 docs should apply, but this information is also below.

The following Ansible variables can be defined to create a route to Elasticsearch:

openshift_logging_es_allow_external=True

(See roles/openshift_logging/README.md for more variables - some of which can be used to customise how the certificates are generated.)

Run the Ansible playbook openshift-logging.yml so that the configuration is applied using these variables. This results in a route which you could allow Grafana, for example, or another application to utilise.

Remote Authentication Against Elasticsearch in OCP 3.6


In order to log in to Elasticsearch remotely the request must contain two HTTP headers: 
 X-Proxy-Remote-User: $user
 Authorization: Bearer $token

The user must have access to the project in order to be able to access to the logs. For example,

  • Create a service account:
 $ oc create sa grafana -n logging
 $ oadm policy add-cluster-role-to-user cluster-reader -n logging -z grafana
  • Get the token of this ServiceAccount like so:
 $ oc sa get-token -n logging grafana

Now, from the outside the Elasticsearch cluster, authentication should be possible:

 $ curl  -kv -H "X-Proxy-Remote-User: grafana" -H "Authorization: Bearer $token" https://elasticsearch.apps.example.com/project.logging.* | python -mjson.tool

Remember to replace $token with your token and the URL and namespace name with your own.

Remote Authentication Against Elasticsearch in OCP 3.5


If you are using OCP 3.5 and wish to authenticate against Elasticsearch, you have two options:
  1. Using Elasticsearch certificates which has its security concerns if the key is compromised.
  2. Use Kibana to query Elasticsearch. This uses a similar approach but you use the Kibana endpoint instead. The below example below uses the $token from a ServiceAccount (see the previous section, above):
$ curl -kv -H "Authorization: Bearer $token" https://`oc get svc logging-kibana -n logging -o jsonpath='{.spec.clusterIP}'`/elasticsearch/.all/_search?sort=@timestamp:desc\&pretty

Querying Elasticsearch through Kibana will perform the authentication at Kibana-proxy just checking whether the user/ServiceAccount can access to that project and then forward the query to Elasticsearch using the Kibana certificates. That means that the Kibana role in SearchGuard will be used which is limited to searching data from indices/aliases.

References

Root Cause

Documentation is available starting in version 3.7. See the subheader, "Expose Elasticsearch as a Route"

SBR
Product(s)
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.