Technology
Strimzi Kafka
Streamline Kafka cluster management with Strimzi. The Strimzi operator automates deployment and operations for Kafka on Kubernetes.
Overview
Overview
Strimzi is an open-source project that provides a Kubernetes Operator to manage Apache Kafka clusters and related components (e.g., Kafka Connect, Kafka MirrorMaker, Kafka Bridge) in a Kubernetes-native way.
Use Cases
- See statistics about Kafka KRaft and Broker.
- Inspect Consumer Groups
- See latency insights.
Setup
Overview
Kafka created by Strimzi operator exposes metrics through a Prometheus endpoint that you can scrape. We recommend to scrape and forward these metrics via an OpenTelemetry Collector. The Strimzi Kafka Exporter exposes additional Prometheus metrics.
Prometheus metrics endpoint
The Kafka metrics are exposed via Kafka broker/controller/exporter HTTP endpoints.
prometheus.io/path: /metrics
prometheus.io/port: "9404"
prometheus.io/scrape: "true"Collector Configuration
The following code snippet shows an OpenTelemetry Collector configuration file.
receivers:
prometheus/kafka:
config:
scrape_configs:
- job_name: "kubernetes-kafka-pod"
kubernetes_sd_configs:
- role: pod
scrape_interval: 30s
scrape_timeout: 5s
relabel_configs:
- action: keep
regex: true
source_labels:
- __meta_kubernetes_pod_annotation_prometheus_io_scrape
- action: drop
regex: true
source_labels:
- __meta_kubernetes_pod_annotation_prometheus_io_scrape_slow
- action: replace
regex: (https?)
source_labels:
- __meta_kubernetes_pod_annotation_prometheus_io_scheme
target_label: __scheme__
- action: replace
regex: (.+)
source_labels:
- __meta_kubernetes_pod_annotation_prometheus_io_path
target_label: __metrics_path__
# Set the address for IPv6 pods based on the prometheus.io/port annotation
- action: replace
regex: (\d+);(([A-Fa-f0-9[]{1,4}::?){1,7}[A-Fa-f0-9[]{1,4})
replacement: "[$$2]:$$1"
source_labels:
- __meta_kubernetes_pod_annotation_prometheus_io_port
- __meta_kubernetes_pod_ip
target_label: __address__
# Set the address for IPv4 pods based on the prometheus.io/port annotation
- action: replace
regex: (\d+);((([0-9[]+?)(\.|$$)){4})
replacement: $$2:$$1
source_labels:
- __meta_kubernetes_pod_annotation_prometheus_io_port
- __meta_kubernetes_pod_ip
target_label: __address__
- action: labelmap
regex: __meta_kubernetes_pod_annotation_prometheus_io_param_(.+)
replacement: __param_$$1
- action: labelmap
regex: __meta_kubernetes_pod_label_(.+)
- action: replace
source_labels:
- __meta_kubernetes_namespace
target_label: namespace
- action: replace
source_labels:
- __meta_kubernetes_pod_name
target_label: pod
# Drop targets that are in non-running phases: Pending, Succeeded, Failed, Completed
- action: drop
regex: Pending|Succeeded|Failed|Completed
source_labels:
- __meta_kubernetes_pod_phase
# Add node label based on the Kubernetes pod node name
- action: replace
source_labels:
- __meta_kubernetes_pod_node_name
target_label: node
processors:
batch:
exporters:
otlp/dash0:
endpoint: "{{endpoint_otlp_grpc}}"
headers:
Authorization: "Bearer {{token}}"
"Dash0-Dataset": "{{dataset}}"
service:
pipelines:
metrics:
receivers: [prometheus/kafka]
processors: [batch]
exporters: [otlp/dash0]Collector Deployment
Learn how to deploy the collector within our OpenTelemetry collector integration documentation.

