The OpenTelemetry Collector is a powerful and flexible tool for handling telemetry, but most teams aren't starting from a blank slate.

In many environments, log pipelines built on Fluentd or Fluent Bit have been running reliably for years and are deeply embedded in existing workflows.

Shutting down these pipelines and replacing every agent in one sweep is rarely practical. This is where the Fluent Forward Receiver becomes essential.

It acts as a bridge between existing Fluent-based infrastructure and an OpenTelemetry pipeline, allowing the Collector to ingest logs using Fluent's native forward protocol.

It's also often the simplest and most reliable path for getting logs from Docker containers into an OpenTelemetry pipeline.

In this guide, we'll move past the sparse official documentation to show how to configure this receiver in an OpenTelemetry pipeline.

What is the Fluent Forward Receiver?

The fluentforward receiver is a TCP server that accepts log events using the Fluent Forward Protocol.

This is the same protocol used by the forward output plugin in both Fluentd and Fluent Bit, which makes it directly compatible with most existing Fluent-based logging setups.

Why is this useful?

Incremental migration: You can redirect your existing Fluent Bit agents to the OpenTelemetry Collector instead of a Fluentd aggregator, without changing the agents themselves.
Pipeline consolidation: It allows you to bring logs from legacy Fluentd infrastructure and modern OTLP-based systems into a single, unified processing pipeline.
Sidecar friendly: It supports Unix domain sockets, which makes it efficient for sidecar patterns where applications write to a local socket and the Collector reads from it.

Quick start: basic configuration

To get started, you need to add the fluentforward receiver to your configuration and add it to a logs pipeline:

yaml
12345678910111213141516
# otelcol.yaml
receivers:
  fluentforward:
    # The endpoint to listen on.
    # 0.0.0.0 allows connections from any interface.
    endpoint: 0.0.0.0:8006

exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    logs:
      receivers: [fluentforward]
      exporters: [debug]

Once this is running, the Collector effectively acts as a Fluentd node. When using TCP (as shown above), the receiver automatically spins up a UDP listener on the same port to handle heartbeat messages, which is a requirement of the Fluent Forward specification.

At this point, you can configure any Fluent Bit or Fluentd agent to forward logs to the Collector. Point the agent's forward output at the same host and port, and the Collector will receive the records exactly as it would in a traditional Fluent stack.

For example, a simple Fluent Bit configuration might look like this:

ini
123456789101112131415161718
# fluent-bit.conf
[SERVICE]
    Flush        1
    Daemon       Off
    Log_Level    info

[INPUT]
    Name         dummy
    Dummy        {"message": "test log from fluent-bit", "service": "demo"}
    Rate         1

[OUTPUT]
    Name         forward
    Match        *
    Host         otelcol
    Port         8006
    # Make sure TLS is off. Fluent Forward doesn't support it
    tls          Off

Once the agent is sending data, you should see log records appear in the Collector's debug output:

text
123456789101112
LogRecord #0
ObservedTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2025-11-17 06:43:14.581469881 +0000 UTC
SeverityText:
SeverityNumber: Unspecified(0)
Body: Str(test log from fluent-bit)
Attributes:
     -> service: Str(demo)
     -> fluent.tag: Str(dummy.0)
Trace ID:
Span ID:
Flags: 0

This confirms that the Fluent Forward receiver is working and that the Collector is correctly parsing and normalizing the incoming events.

Configuring fluent forward for real workloads

The basic configuration is fine for a quick test, but you'll usually want to tighten transport, reliability, and integration with sidecars.

1. Using unix sockets for local agents

If your log source, such as a Fluent Bit sidecar, runs on the same node as the Collector, a Unix domain socket is often a better fit than TCP.

It avoids port management, stays on the local host, and keeps the traffic inside a clear security boundary.

yaml
12345
# otelcol.yaml
receivers:
  fluentforward:
    # Use the unix:// prefix to specify the socket path
    endpoint: unix:///var/run/fluent.sock

In your Fluent Bit configuration, replace Host and Port with the path to the Unix socket:

ini
12345
# fluent-bit.conf
[OUTPUT]
    Name         forward
    Match        *
    Unix_Path    /socket/fluent.sock

Note: The user running the OpenTelemetry Collector must have read and write permissions on the socket file or on the directory where it will be created or you may see an error that looks like this:

text
1
2025/11/17 06:50:18 collector server run finished with error: cannot start pipelines: failed to start "fluentforward" receiver: listen unix /socket/fluent.sock: bind: permission denied

2. Enabling acknowledgments for safer delivery

The Fluent Forward protocol supports an acknowledgment mode, where the sender waits for a reply before discarding a batch of logs. The fluentforward receiver understands these acknowledgments and responds accordingly.

When you use Fluent Bit as the sender, turn this on so that temporary network issues do not silently drop data:

ini
1234567
# fluent-bit.conf
[OUTPUT]
    Name                  forward
    Match                 *
    Host                  otel-collector-host
    Port                  8006
    Require_ack_response  True

With acknowledgments enabled, Fluent Bit will retry chunks that are not confirmed, giving you at-least-once delivery semantics between the agent and the Collector.

3. Matching Fluentd-level reliability with persistent queues

One of the biggest surprises for teams moving from Fluentd to the OpenTelemetry Collector is how buffering works.

Fluentd usually writes its buffers to disk so that if the process restarts, the buffered chunks are still there and will be retried.

The Collector behaves differently. By default, it buffers in memory which means if process crashes or the pod is restarted, any unflushed data is lost.

To get the same durability guarantees you are used to with Fluentd, you need to enable the file_storage extension and point your exporter's queue at it.

yaml
1234
# otelcol.yaml
extensions:
  file_storage:
    directory: /var/lib/otelcol/file_storage

In your configured exporter, enable the sending queue and bind it to the file_storage extension so it survives restarts:

yaml
123456789
# otelcol.yaml
exporters:
  otlp:
    endpoint: my-backend.example.com:4317
    sending_queue:
      enabled: true
      # This is what makes the queue persistent
      storage: file_storage
      queue_size: 10000

The Collector will ignore the extension unless it is explicitly enabled:

yaml
12345678
# otelcol.yaml
service:
  extensions: [file_storage] # <-
  pipelines:
    logs:
      receivers: [fluentforward]
      processors: [batch]
      exporters: [otlp]

With this setup in place, if the Collector crashes, restarts, or is OOM-killed, it reloads any unsent log batches from /var/lib/otelcol/file_storage and resumes delivery automatically.

This brings you much closer to Fluentd's durability model and prevents silent data loss during node interruptions or rolling deployments.

Working around the lack of TLS support

There is one important limitation with the fluentforward receiver that you must be aware of: it does not support TLS.

The documentation mentions this in passing, but the security impact is significant. If you expose this endpoint on a shared or untrusted network, every log record travels in cleartext. If your logs contain user data, tokens, or internal identifiers, that is not acceptable.

You should treat this receiver as a "plain TCP on a trusted link only" component and add encryption around it.

Some practical ways to contain the risk

Keep the receiver on a mesh that provides mutual TLS between workloads, for example Istio or Linkerd. The Collector and the Fluent Bit or Fluentd agents speak plain Fluent Forward, while the mesh sidecars handle encryption and identity.
Bind the receiver only on trusted addresses, such as 127.0.0.1, a private VPC subnet or the Node IP (in Kubernetes). Avoid exposing it through public load balancers or routable IPs that cross security boundaries.
Where possible, use a Unix domain socket on the same node instead of TCP. The application or sidecar writes to a local socket, the Collector reads from it, and the traffic never leaves the host. This reduces the blast radius and simplifies your threat model.

How Fluent data maps to OpenTelemetry logs

A common point of confusion when migrating is understanding how a Fluentd or Fluent Bit event becomes an OpenTelemetry LogRecord.

The Fluent Forward receiver performs a straightforward translation, but knowing the exact mapping makes it much easier to reason about downstream processors and exporters.

Here's what happens when the receiver accepts an event:

The event tag is preserved as an attribute on the log record. The default key is fluent.tag.
The Fluent event time becomes the Timestamp field of the LogRecord.
Unless your sender adds severity information in a dedicated field and you map it, SeverityText and SeverityNumber stay unset.

Checking the mapping with the debug exporter

The debug exporter is the simplest way to see exactly how your data arrives in the Collector as it prints the raw OpenTelemetry representation of each event.

For example, the following input from Fluent Bit:

json
1
{ "key": "value", "user_id": 123, "message": "A log message" }

Yields the following log in an OpenTelemetry Pipeline (assuming verbosity: detailed):

text
12345678910111213
LogRecord #0
ObservedTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2025-11-17 07:28:49.093622498 +0000 UTC
SeverityText:
SeverityNumber: Unspecified(0)
Body: Str(A log message)
Attributes:
     -> key: Str(value)
     -> user_id: Int(123)
     -> fluent.tag: Str(dummy.0)
Trace ID:
Span ID:
Flags: 0

With the data in this shape, you can enrich or restructure it before export with processors like Attributes, Resource or Transform.

Troubleshooting: why your logs are not arriving

If your Fluent Bit or Fluentd agents are sending data but nothing shows up in the Collector, walk through these checks.

1. Handshake settings on Fluentd

The fluentforward receiver does not implement the Fluentd secure handshake phase. If your Fluentd sender is configured to require a handshake or shared key authentication, the connection will fail.

Make sure any security, shared_key, or handshake related options on the sender are disabled or set to use a plain forward connection.

2. TCP and UDP connectivity

When you configure a TCP endpoint, the receiver also opens a UDP port on the same number to handle heartbeat messages. For example, if you listen on 0.0.0.0:8006:

TCP 8006 is used for log traffic.
UDP 8006 is used for heartbeats.

Firewalls or security groups must allow both directions for both TCP and UDP on that port, otherwise senders may treat the endpoint as unhealthy and stop forwarding.

3. Compression and "packed forward" format

The receiver supports packed forward messages, including compressed variants. If you see errors about garbled payloads or invalid format in the Collector logs, verify the sender configuration:

Use the standard packed forward format.
If compression is enabled, make sure it uses a method compatible with the Go compress/gzip or zlib libraries.
Temporarily disable compression to confirm basic connectivity, then reintroduce it once the plain setup works.

If these checks look correct and you still do not see logs, turn on the Collector's debug exporter and increase log verbosity on the sender. This usually reveals whether the issue is connection level, protocol level, or a simple tag or match misconfiguration.

Final thoughts

The Fluent Forward Receiver lets you bring the OpenTelemetry Collector into your environment without rewriting or redeploying every Fluent Bit or Fluentd agent on day one. That breathing room matters as it means you can evolve your logging architecture without breaking what already works.

Once your logs land in the Collector, the next step is putting them to work alongside your traces and metrics. At Dash0, that's exactly what we focus on. We bring all three signals together, automatically linking logs from any source so you can understand issues in seconds instead of digging through disconnected streams of data.

Get more out of your observability data by starting your free Dash0 trial today.

Bridging Fluentd Pipelines to OpenTelemetry with Fluent Forward