Identify the namespaces that are producing the most amount of Logs in RHOCP4

• This article will help in Identification of the Namespaces/Project that are generating the most amount of Logs.
• The Storage is getting over utilised, How to find out which Namespaces/Project is using the most amount of space?

Prerequisites

• Red Hat OpenShift Logging (RHOL) version 5.8 or higher.
• Label openshift.io/cluster-monitoring: "true" should be set on openshift-logging namespace.
• LogFileMetricExporter Must be deployed in openshift-logging namespace.

How to setup LogFileMetricExporter

• Starting from Red Hat OpenShift Logging (RHOL) version 5.8, Log File Metric Exporter (LFME) is not deployed by default, It needs to be created as per the documentation.

How to Check Dashboards

• Once the LFME is deployed in the cluster, The dashboard under Observe --> Dashboards --> Logging/Collection will display the metrics related to the logs.

  

• To check which are the highest log producing namespaces/project, Scroll down to the section Produced Logs

  

• Clicking on Inspect in the above dashboards will give the query that is used to fetch the result

  • Top producing Container:

  $ topk(10, round(rate(log_logged_bytes_total[5m])))  --> 10 refers to top 10, Values can be changed as per requirement.
  

  • Top producing containers in last 24 hours:

  $ topk(10, sum(increase(log_logged_bytes_total[24h])) by (exported_namespace,  podname, containername))     --> 10 refers to top 10, Values can be changed as per requirement.