Configure log retention per namespace for OpenShift Elasticsearch log store

Environment

• Red Hat OpenShift Container Platform (RHOCP)
  • 4.9+
• Red Hat OpenShift Logging (RHOL)
  • 5.5+
• Elasticsearch operator

Issue

• Is it possible to configure log retention for specific namespace in OpenShift Logging?
• How to configure namespace level log retention?
• Configure log retention per namespace in OpenShift 4.

Resolution

Starting with RHOL 5.5, it is possible to configure log retention for specific namespaces when using Elasticsearch.
To configure this, edit clusterlogging custom resource and append below config in logStore section:

$ oc edit clusterlogging instance -n openshift-logging
---------- Output Omitted ----------
  logStore:
    type: "elasticsearch"
    retentionPolicy: 
      application:
        maxAge: 1d
        pruneNamespacesInterval: 60m          //  This value is 30 minutes by default
        namespaceSpec:
        - namespace: my-workload-ns
          minAge: 5h
      infra:
        maxAge: 7d
      audit:
        maxAge: 7d
    elasticsearch:
---------- Output Omitted ----------

More details about configured parameters

• pruneNamespacesInterval: This is optional and is responsible for configuring frequency at which prune-namespace job will be triggered. Once this parameter is configured, a new cronjob is created named elasticsearch-im-prune-app (for application indices) and elasticsearch-im-prune-infra (for infra indices).
• minAge: Deletes the record matching the namespaces which are older than this minAge.
• namespace: Target namespace to delete logs older than minAge. It can be defined for one namespace or a prefix (e.g., "abc-" covers all namespaces with this prefix).

Note: These configs are applicable for application and infra aliases.

Root Cause

Starting with RHOL 5.5, it is possible to configure log retention for specific namespaces when using Elasticsearch.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.