Vector sends duplicated the logs when log forwarding to syslog in RHOL

Environment

• Red Hat OpenShift Container Platform (RHOCP)
  • 4
• Red Hat OpenShift Logging (RHOL)
  • 5.8
  • 5.9
  • 6.0
  • 6.1
  • 6.2
  • 6.3
• Vector

Issue

• When log forwarding to syslog, a single log line produced is log forwarded duplicated to the syslog server
• After migrating from Fluentd to Vector, the number of logs received in the syslog server has increased significantly

Resolution

This issue has been reported to Red Hat engineering. It is being tracked in Bug:

For more information, please open a This content is not included.new support case with Red Hat Support.

Root Cause

Vector duplicated a log message when forwarding logs to syslog, leading to redundant data in the output.

Diagnostic Steps

1. In the next steps is assumed that the namespace: openshift-logging and the name of the custom resource (CR) is instance. Let's set the variables:

   $ cr="instance"
   $ ns="openshift-logging"
   

2. Verify that the collector type is Vector. This verification is only needed if the RHOL version is v5 as in v6 Fluentd is not available:

   $ oc get clusterlogging $cr -n $ns -o jsonpath='{.spec.collection.type}'
   vector
   

3. Verify that configured the Red Hat Logging stack to log forward to a syslog server:

       // In Logging 5
       $ oc get clusterlogforwarder $cr -n $ns -o yaml
   
       // In Logging 6
       $ oc get obsclf $cr -n $ns -o yaml
   

4. Verify in the syslog server side that a single log line produced in the OpenShift cluster is log forwarded by the collector as two log messages.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.