How to query from the command line prometheus statistics when monitoring own services
Environment
- Red Hat Openshift Container Platform 4.X
Issue
- need to access metrics from outside the cluster for custom applications
Resolution
-
Follow documentation to This page is not included, but the link has been rewritten to point to the nearest parent document.Enabling monitoring for user defined projects
-
Once the deployment of your own service done, you can access monitoring data from outside the cluster with thanos-querier route
-
Create service account with rights to access user workload prometheus data:
oc new-project test
oc create sa user-monitoring-viewer
- Assign rights to this service account:
oc adm policy add-cluster-role-to-user cluster-monitoring-view system:serviceaccount:test:user-monitoring-viewer
- Extract token:
export TOKEN=$(oc create token user-monitoring-viewer -n test)
- extract your route host endpoint to query metrics:
export THANOS_QUERIER_HOST=`oc get route thanos-querier -n openshift-monitoring -o json | jq -r '.spec.host'`
- query the metrics of your own services in the command line like this:
curl -X GET -k "https://$THANOS_QUERIER_HOST/api/v1/query?query=version%7Bnamespace%3D%22ns1%22%7D" -H "Authorization: Bearer $TOKEN"
You should see, from the former example, the version metric exposed by your application:
{"status":"success","data":{"resultType":"vector","result":[{"metric":{"__name__":"version","endpoint":"web","instance":"10.128.3.186:8080","job":"prometheus-example-app","namespace":"ns1","pod":"prometheus-example-app-6456bd9798-cvf4l","prometheus":"openshift-user-workload-monitoring/user-workload","service":"prometheus-example-app","version":"v0.4.2"},"value":[1721235397.557,"1"]}]}}
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.