Logging extensions are part of the Banzai Cloud One Eye observability system, and are also a standalone operator. Logging extensions were specifically developed to solve the problems of enterprises:

  • Collecting Kubernetes events to provide insight into what is happening inside a cluster, such as decisions made by the scheduler, or why some pods were evicted from the node.
  • Collect logs from the nodes like kubelet logs.
  • Collect logs from files on the nodes, for example, audit logs, or the systemd journal.

You can configure the extensions in the One Eye custom resource configuration. It’s also a standalone kubernetes operator.

Follow this guide to install the One Eye command line tool.

Kubernetes Event Tailer 🔗︎

Kubernetes events are objects that provide insight into what is happening inside a cluster, such as what decisions were made by the scheduler or why some pods were evicted from the node.

Example: configuration Kubernetes event tailer 🔗︎

kubectl apply -f - <<EOF
apiVersion: logging-extensions.banzaicloud.io/v1alpha1
kind: EventTailer
metadata:
  name: sample
spec:
  controlNamespace: default
EOF

Example: configuration Kubernetes event tailer with PVC 🔗︎

kubectl apply -f - <<EOF
apiVersion: logging-extensions.banzaicloud.io/v1alpha1
kind: EventTailer
metadata:
  name: sample
spec:
  controlNamespace: default
  positionVolume:
    pvc:
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
        volumeMode: Filesystem
EOF

Configuration options 🔗︎

Variable Name Type Required Default Description
controlNamespace string Yes - The resources of Eventtailer will be placed into this namespace
positionVolume volume.KubernetesVolume No - Volume definition for tracking fluentbit file positions (optional)
workloadMetaOverrides *types.MetaBase No - Override metadata of the created resources
workloadOverrides *types.PodSpecBase No - Override podSpec fields for the given statefulset

Example: Configure logging Flow to route logs from an event tailer 🔗︎

The following example configures a flow to route the previously created sample-eventtailer EventTailer’s log.

kubectl apply -f - <<EOF
apiVersion: logging.banzaicloud.io/v1beta1
kind: Flow
metadata:
  name: eventtailer-flow
  namespace: default
spec:
  filters:
  - tag_normaliser: {}
  match:
  # keeps data matching to label, the rest of the data will be discarded by this flow implicitly
  - select:
      labels:
        app.kubernetes.io/name: sample-event-tailer
  outputRefs:
    - sample-output
EOF

Kubernetes Host Tailers 🔗︎

Tailing logs from the nodes like kubelet, audit logs or from the systemd journal.

Kubernetes Systemd tailer 🔗︎

Tail logs from the systemd journal. Define one or more systemd tailers in the Observer configuration.

Example: configuration Systemd tailer 🔗︎

kubectl apply -f - <<EOF
apiVersion: logging-extensions.banzaicloud.io/v1alpha1
kind: HostTailer
metadata:
  name: systemd-hosttailer-sample
spec:
  systemdTailers:
    - name: my-systemd-tailer
      disabled: false
      maxEntries: 100
      systemdFilter: kubelet.service
EOF

Systemd tailer configuration options 🔗︎

Variable Name Type Required Default Description
name string Yes - Name for the tailer
path string No - Override systemd log path
disabled bool No - Disable component
systemdFilter string No - Filter to select systemd unit example: kubelet.service
maxEntries int No - Maximum entries to read when starting to tail logs to avoid high pressure
containerOverrides *types.ContainerBase No - Override container fields for the given tailer

Kubernetes Host File tailer 🔗︎

Tail logs from the node’s host filesystem. Define one or more file tailers in the HostTailer configuration.

Example: configuration host File Tailer 🔗︎

kubectl apply -f - <<EOF
apiVersion: logging-extensions.banzaicloud.io/v1alpha1
kind: HostTailer
metadata:
  name: file-hosttailer-sample
spec:
  fileTailers:
    - name: nginx-access
      path: /var/log/nginx/access.log
      disabled: false
EOF

File Tailer configuration options 🔗︎

Variable Name Type Required Default Description
name string Yes - Name for the tailer
path string No - Path to the loggable file
disabled bool No - Disable tailing the file
containerOverrides *types.ContainerBase No - Override container fields for the given tailer

Example: Configure logging Flow to route logs from a Hosttailer 🔗︎

The following example uses the flow’s match term to listen the previously created file-hosttailer-sample Hosttailer’s log.

kubectl apply -f - <<EOF
apiVersion: logging.banzaicloud.io/v1beta1
kind: Flow
metadata:
  name: hosttailer-flow
  namespace: default
spec:
  filters:
  - tag_normaliser: {}
  # keeps data matching to label, the rest of the data will be discarded by this flow implicitly
  match:
  - select:
      labels: 
        app.kubernetes.io/name: file-hosttailer-sample
      # there might be a need to match on container name too (in case of multiple containers)
      container_names:
        - nginx-access
  outputRefs:
    - sample-output
EOF

Example: configuration Kubernetes host tailer with multiple tailers 🔗︎

kubectl apply -f - <<EOF
apiVersion: logging-extensions.banzaicloud.io/v1alpha1
kind: HostTailer
metadata:
  name: multi-sample
spec:
  # list of File tailers
  fileTailers:
    - name: nginx-access
      path: /var/log/nginx/access.log
    - name: nginx-error
      path: /var/log/nginx/error.log
  # list of Systemd tailers
  systemdTailers:
    - name: my-systemd-tailer
      maxEntries: 100
      systemdFilter: kubelet.service
EOF

Configuration options 🔗︎

Variable Name Type Required Default Description
fileTailers []FileTailer No - List of file tailers
systemdTailers []SystemdTailer No - List of systemd tailers
enableRecreateWorkloadOnImmutableFieldChange bool No - EnableRecreateWorkloadOnImmutableFieldChange enables the operator to recreate the
fluentbit daemonset and the fluentd statefulset (and possibly other resource in the future)
in case there is a change in an immutable field
that otherwise couldn’t be managed with a simple update.
workloadMetaOverrides *types.MetaBase No - Override metadata of the created resources
workloadOverrides *types.PodSpecBase No - Override podSpec fields for the given daemonset

Example: Configure logging Flow to route logs from a Hosttailer 🔗︎

The following example uses the flow’s match term to listen the previously created file-hosttailer-sample Hosttailer’s log.

kubectl apply -f - <<EOF
apiVersion: logging.banzaicloud.io/v1beta1
kind: Flow
metadata:
  name: hosttailer-flow
  namespace: default
spec:
  filters:
  - tag_normaliser: {}
  # keeps data matching to label, the rest of the data will be discarded by this flow implicitly
  match:
  - select:
      labels: 
        app.kubernetes.io/name: file-hosttailer-sample
      # there might be a need to match on container name too (in case of multiple containers)
      container_names:
        - nginx-access
  outputRefs:
    - sample-output
EOF

Example: configuration Kubernetes host tailer with multiple tailers 🔗︎

kubectl apply -f - <<EOF
apiVersion: logging-extensions.banzaicloud.io/v1alpha1
kind: HostTailer
metadata:
  name: multi-sample
spec:
  # list of File tailers
  fileTailers:
    - name: nginx-access
      path: /var/log/nginx/access.log
    - name: nginx-error
      path: /var/log/nginx/error.log
  # list of Systemd tailers
  systemdTailers:
    - name: my-systemd-tailer
      maxEntries: 100
      systemdFilter: kubelet.service
EOF

Example: Setting up custom priority 🔗︎

Create your own custom priority class in Kubernetes. Set its value between 0 and 2000000000.

Priority Hints:

  • 0 is the default priority
  • To change the default priority, set the globalDefault key
  • 2000000000 and above are reserved for kubernetes system
  • PriorityClass is a non-namespaced object
kubectl apply -f - <<EOF
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: hosttailer-priority
value: 1000000
globalDefault: false
description: "This priority class should be used for hosttailer pods only."
EOF

Now you can use your private priority class name to start hosttailer/eventtailer as it shown below:

kubectl apply -f -<<EOF
apiVersion: logging-extensions.banzaicloud.io/v1alpha1
kind: HostTailer
metadata:
  name: priority-sample
spec:
  controlNamespace: default
  # Override podSpecBase variables here
  workloadOverrides:
    priorityClassName: hosttailer-priority
  fileTailers:
    - name: nginx-access
      path: /var/log/nginx/access.log
    - name: nginx-error
      path: /var/log/nginx/error.log
EOF

Configuration options 🔗︎

Variable Name Type Required Default Description
fileTailers []FileTailer No - List of file tailers
systemdTailers []SystemdTailer No - List of systemd tailers
enableRecreateWorkloadOnImmutableFieldChange bool No - EnableRecreateWorkloadOnImmutableFieldChange enables the operator to recreate the
fluentbit daemonset and the fluentd statefulset (and possibly other resource in the future)
in case there is a change in an immutable field
that otherwise couldn’t be managed with a simple update.
workloadMetaOverrides *types.MetaBase No - Override metadata of the created resources
workloadOverrides *types.PodSpecBase No - Override podSpec fields for the given daemonset

Advanced configuration overrides 🔗︎

MetaBase 🔗︎

Variable Name Type Required Default Description
annotations map[string]string No -
labels map[string]string No -

PodSpecBase 🔗︎

Variable Name Type Required Default Description
tolerations []corev1.Toleration No -
nodeSelector map[string]string No -
serviceAccountName string No -
affinity *corev1.Affinity No -
securityContext *corev1.PodSecurityContext No -
volumes []corev1.Volume No -
priorityClassName string No -

ContainerBase 🔗︎

Variable Name Type Required Default Description
resources *corev1.ResourceRequirements No -
image string No -
pullPolicy corev1.PullPolicy No -
command []string No -
volumeMounts []corev1.VolumeMount No -
securityContext *corev1.SecurityContext No -