How to migrate Fluentd to Vector in Red Hat OpenShift Logging 5.5+ versions ?

Updated

The following document describes how to migrate the OpenShift Logging collection service from Fluentd to Vector. Starting OpenShift Logging 5.6, Vector replaces Fluentd as the default Collector. In Logging 5.6, Fluentd is deprecated and is planned to be removed in a future release. Red Hat will provide bug fixes and support for this feature during the current release lifecycle, but this feature will no longer receive enhancements and will be removed.

Prerequisites

  • Installed Red Hat OpenShift Logging Operator (current stable v5.5+)
  • Review features (supported and unsupported) comparison between Vector and Fluentd before performing a migration to Vector.

Note:

  • To migrate default log store from Elasticsearch to Loki in Red Hat OpenShift Logging follow the article.
  • To migrate the collector from Fluentd to Vector reducing the number of logs duplicated follow the article

Current Stack

Assuming the current stack looks like the below that represents basic log collection only. The config might vary regarding resources/nodes/tolerations/selectors.

apiVersion: "logging.openshift.io/v1"
kind: "ClusterLogging"
metadata:
  name: "instance"
  namespace: "openshift-logging"
spec:
  managementState: "Managed"
  collection:
    logs:
      type: "fluentd"  
      fluentd:
        resources:
          limits: 
            memory: 736Mi
          requests:
            cpu: 100m
            memory: 736Mi
----- Output Omitted -----

Stack Migrated to Vector

Edit the clusterlogging instance custom resource to use vector as log collection type. Allocated resources can be customized as shown in below config. To configure CPU and memory limits for Vector, follow this article.

$ oc edit clusterlogging instance -n openshift-logging

apiVersion: "logging.openshift.io/v1"
kind: "ClusterLogging"
metadata:
  name: "instance"
  namespace: "openshift-logging"
spec:
  managementState: "Managed"
  collection:
    type: vector
----- Output Omitted -----
Category
Components
Article Type