Ingress-nginx is an ingress controller. It sits at the edge of your cluster, routing external requests to pods that would otherwise not be reachable from the outside. Ingress controllers are a critical part of most Kubernetes setups, yet often remain a black box. While Ingress-NGINX now offers native OpenTelemetry tracing, its metrics and logs still live in separate worlds. This post shows how to combine all three signals using the OpenTelemetry Collector and Dash0, turning Ingress-NGINX into a fully observable part of your platform.
Ingress-nginx is one of the most widely deployed ingress controllers in Kubernetes. It acts as the front door to your cluster, accepting external requests, terminating TLS, and routing traffic to the right services. Because it sits at the very edge, it is also one of the most critical components for reliability and performance. If the ingress slows down, everything behind it feels slower. If the ingress misroutes or fails, the user never reaches your services at all.
Yet for many teams, the ingress layer remains a black box. You might scrape a few Prometheus metrics or check log files when debugging, but that still leaves unanswered questions. Which routes are producing errors most often? How much latency does the ingress itself add before handing off to a backend? How do access logs tie into the request traces you see in your observability platform? And this is such a missed opportunity! The fact that the ingress controller sits between incoming requests and the pods serving them makes it the perfect observability linchpin!
That’s where OpenTelemetry helps. Ingress-nginx has native support for distributed tracing, giving you spans for every request that passes through the controller. Metrics and logs are not OpenTelemetry-native - they are still exposed via a Prometheus endpoint and written as traditional NGINX access logs. But with the OpenTelemetry Collector, you can ingest those signals, enrich them, and correlate them with the traces. The result is a unified picture of ingress behavior that combines high-level trends with per-request detail. In this post, we’ll walk through how to set it up, and we’ll use Dash0 to show how these signals come together in practice.
Enabling tracing in ingress-nginx
Tracing is the one OpenTelemetry signal ingress-nginx supports natively. The controller includes an OpenTelemetry module that can emit spans directly via the OpenTelemetry Protocol (OTLP). If a client sends trace context headers, ingress-nginx will trust and propagate them. If not, it starts a new one at the edge. This means the ingress can either join an existing trace or become the root span of a trace, depending on what sits in front of it. If you’re running behind a cloud load balancer or API gateway that already injects trace headers, ingress-nginx continues that trace. If ingress-nginx is the first hop, its span is the root - the entry point of your distributed trace.
You can enable tracing either through the controller’s Helm values or ConfigMap, or by applying the official nginx.ingress.kubernetes.io/enable-opentelemetry annotation on individual Ingress resources if you only want tracing on selected routes (docs). Using annotations is useful when you want a gradual rollout of tracing rather than enabling it cluster-wide.
Here’s an example Helm values configuration to enable tracing for the whole controller:
12345678controller:config:enable-opentelemetry: "true"otlp-collector-host: "opentelemetry-collector.opentelemetry.svc"otlp-collector-port: "4317"otel-service-name: "ingress-nginx"otel-sampler-ratio: "1.0"otel-sampler: "AlwaysOn"
This points ingress-nginx at an in-cluster OpenTelemetry Collector service on port 4317 (the OTLP gRPC default). The otel-service-name makes the spans easy to identify in Dash0. Sampling is set to “AlwaysOn” for the demo, but you can tune this in production. Once deployed, you’ll see ingress spans in Dash0 carrying attributes like http.method, http.target, http.status_code, and timing information. Because context is propagated, those spans can appear as parents to your service spans, connecting the ingress with the rest of your traces. (note, that the module uses a deprecated version of semantic conventions for HTTP spans; but if you use Dash0, we will automatically upgrade you to the latest & greatest)
Exposing and collecting metrics
Metrics are not emitted as OpenTelemetry data. Ingress-nginx exposes them in Prometheus format on port 10254. These metrics include request counters, latency histograms, response sizes, and NGINX process stats. To bring them into the same observability pipeline, the OpenTelemetry Collector’s Prometheus receiver scrapes the endpoint and ingests the metrics in their native Prometheus shape.
Here’s how you enable metrics and annotate the pods for scraping:
1234567controller:metrics:enabled: truepodAnnotations:prometheus.io/scrape: "true"prometheus.io/port: "10254"prometheus.io/path: "/metrics"
The scraping itself happens in the OpenTelemetry Collector Deployment in the demo, not the DaemonSet. Running it in the Deployment ensures each ingress-nginx pod is scraped only once, avoiding duplication if you run multiple replicas. The Deployment’s Prometheus receiver discovers pods with these annotations and scrapes their metrics.
At a larger scale, you have a few other options. You can run the Prometheus receiver in a DaemonSet and configure it to scrape only the pods scheduled on the same node. This node-affinity style of scraping avoids cross-node traffic and scales horizontally with your cluster - it’s how we handle it in the Dash0 Operator, and it tends to be more efficient. Another option is to use the TargetAllocator built into the OpenTelemetry Operator, which dynamically assigns scrape targets to Collector replicas so that each target is scraped by exactly one instance. Both patterns eliminate duplication while spreading the load across your cluster.
Once in Dash0, you can explore key metrics like nginx_ingress_controller_requests for request volume, or nginx_ingress_controller_response_duration_seconds for latency distributions. Even though the project hasn’t adopted OpenTelemetry metrics, the Collector makes them part of the same telemetry stream. Dash0 dashboards can show throughput, error rates, and p95/p99 latencies, side by side with traces and logs.
Making logs first-class citizens
Logs are also not OpenTelemetry-native. Ingress-nginx produces classic NGINX-style access logs, one line per request. By default, they have no connection to traces. To make them more useful, you can customize the log format to include the OpenTelemetry trace and span IDs:
12345678controller:config:log-format-upstream: >-$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent"$http_referer" "$http_user_agent" $request_length $request_time[$proxy_upstream_name] [$proxy_alternative_upstream_name] $upstream_addr$upstream_response_length $upstream_response_time $upstream_status $req_idtrace_id=$opentelemetry_trace_id span_id=$opentelemetry_span_id
Correlating logs with traces
With these IDs in place, the Collector’s filelog receiver can tail the logs and apply an OTTL transform to set the trace_id and span_id fields on each log record. That way, every log line in Dash0 is linked to the trace it belongs to. You can click from a trace to its log entry, or filter logs by trace ID to see everything related to a specific request.
This correlation is powerful. Imagine you see a 2.5-second span in Dash0. The trace shows it’s from the ingress, but you want the gritty details. With correlation, you can jump into the access log line for that span and see the upstream response time, the backend IP, and the exact request path. Or, if you’re reviewing logs and spot multiple 504s, you can pivot into the trace view to see how those requests behaved end to end.
Logs start out as flat text, but with the trace and span ID stitched in, they become a structured signal that lives alongside metrics and traces.
Deploying the OpenTelemetry Collector
The Collector is the glue that ties the signals together. In the demo, it runs in two forms: a DaemonSet and a Deployment.
Collector as a DaemonSet
The DaemonSet ensures a Collector pod on every node. This local presence allows it to read container logs from the node filesystem. In our setup, the DaemonSet runs the filelog receiver for ingress logs and applies the OTTL transforms that extract trace and span IDs. The DaemonSet then exports enriched log records to Dash0.
Collector as a Deployment
The Deployment plays two roles. First, it provides a central OTLP endpoint for traces. Ingress-nginx is configured to send spans to it. The Deployment runs the OTLP receiver, batches spans, enriches them with Kubernetes metadata via the k8sattributes processor, and exports them to Dash0. Second, the Deployment runs the Prometheus receiver to scrape ingress-nginx metrics. This avoids duplication and centralizes metric collection.
Together, the two Collector modes ensure all three signals reach Dash0: traces directly from ingress-nginx, metrics scraped once centrally, and logs tailed locally with correlation.
Putting it all together in Dash0
With the configuration in place, Dash0 provides a single lens on ingress-nginx behavior. Metrics show request rates, error percentages, and latency distributions. Traces reveal the journey of individual requests, with ingress spans as the root or as part of a longer chain, depending on what sits in front of the controller. Logs, now stitched into traces, give you the raw detail when you need it.
Dash0’s Integration Hub includes an Ingress-NGINX integration with setup instructions and a prebuilt dashboard. The dashboard pulls together the most important metrics so you can see ingress performance at a glance. When issues arise, you can pivot between signals seamlessly: a trace to its log line, a log to its trace, or a metric spike to the requests behind it.
Final thoughts
Ingress-nginx is too important to remain a blind spot. With native tracing support, plus metrics and logs ingested through the OpenTelemetry Collector, you can make it fully observable. The key is correlation: metrics show the trend, traces show the path, and logs show the detail - and together they answer the questions a single signal never could.
In environments without an upstream propagator, ingress spans will be the root of your traces. In environments with load balancers or gateways that inject headers, ingress spans slot into a broader context. Either way, they give you visibility into the edge.
For platform engineers, this closes a gap. You can now monitor the very first hop in your system with the same confidence as the services behind it. Check out the dash0-examples repository for the full demo setup, and the Ingress-NGINX integration on the Dash0 Hub for a ready-made dashboard. With OpenTelemetry and Dash0, the edge of your cluster no longer needs to be a black box.
