Dash0 Raises $35 Million Series A to Build the First AI-Native Observability Platform

Last updated: October 12, 2025

Collecting Prometheus Metrics with the OpenTelemetry Collector

Observability in modern systems often starts with Prometheus. It's the tool most developers and operators rely on to scrape and expose metrics from their applications and infrastructure.

But as teams adopt OpenTelemetry for unified telemetry collection, the question becomes: how do you bring all that Prometheus data into the OpenTelemetry world?

That's where the Prometheus receiver comes in. It lets the OpenTelemetry Collector act like a Prometheus server by scraping any Prometheus-compatible endpoint then converting the metrics into the OpenTelemetry Protocol (OTLP), and sends them through the Collector's pipelines for further processing and export.

In other words, you get all the power of Prometheus's scraping and discovery mechanisms, combined with OpenTelemetry's flexibility and interoperability. This guide walks you through how it works in practice, from setting up simple scrapes to scaling collection efficiently in production.

Quick start: scraping your first target

A good way to get familiar with the Prometheus receiver is to start small and have it scrape the Collector's own metrics endpoint. This gives you a simple, self-contained "hello world" setup.

First, make sure the Collector exposes its own metrics in Prometheus format. You can do that in the service::telemetry::metrics section of your configuration:

yaml
12345678910111213141516
# otelcol.yaml
service:
telemetry:
metrics:
readers:
- pull:
exporter:
prometheus:
host: "0.0.0.0"
port: 8888
pipelines:
metrics:
receivers: [prometheus]
processors: [batch]
exporters: [debug] # Use the debug exporter to view the output

Next, configure the Prometheus receiver to scrape that endpoint:

yaml
123456789
# otelcol.yaml
receivers:
prometheus:
config:
scrape_configs:
- job_name: otel-collector
scrape_interval: 10s
static_configs:
- targets: ["0.0.0.0:8888"] # Scrape the Collector's own metrics

Keep in mind that this setup is only for demonstration, since the Collector can already expose metrics in OTLP format using its otlp exporter.

When you run the Collector with this configuration, the debug exporter will print the scraped metrics to your console.

You'll see OTLP-formatted metrics that were scraped from the Prometheus endpoint, similar to the example below:

text
1234567891011121314
[...]
Metric #4
Descriptor:
-> Name: otelcol_process_cpu_seconds_total
-> Description: Total CPU user and system time in seconds [alpha]
-> Unit:
-> DataType: Sum
-> IsMonotonic: true
-> AggregationTemporality: Cumulative
NumberDataPoints #0
StartTimestamp: 2025-10-12 03:44:48.252 +0000 UTC
Timestamp: 2025-10-12 03:45:58.041 +0000 UTC
Value: 0.360000
[...]

Understanding the Prometheus receiver configuration

The Prometheus receiver's strength comes from how closely it mirrors Prometheus's own configuration model. You can take the same configuration that would normally live in a prometheus.yml file and place it under the config: key in your Collector setup. This makes it simple to migrate existing Prometheus setups.

At the center of this configuration are the scrape_configs and metric_relabel_configs sections. Together, they define what gets scraped, how it's labeled, and which metrics make it into your pipeline.

scrape_configs

This section defines what the receiver scrapes and how it does it. Each entry represents a scrape job with its own targets and parameters.

One of the most common real-world examples is discovering and scraping Kubernetes pods that have been annotated for Prometheus.

Here's what that might look like:

yaml
12345678910111213141516171819202122232425
# otelcol.yaml
receivers:
prometheus:
config:
scrape_configs:
# Discover Kubernetes pods that have a Prometheus scrape annotation
- job_name: "k8s-annotated-pods"
kubernetes_sd_configs:
- role: pod
# Relabeling defines which pods to include and how to build their targets
relabel_configs:
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
action: keep
regex: true
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels: [ __address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
target_label: __address__
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $$1:$$2

Note that the OpenTelemetry Collector uses $ for environment variable substitution. If you use capture groups in relabeling rules (for example $1), you must escape them as $$1. Otherwise, the Collector will interpret $1 as an environment variable.

metric_relabel_configs

While relabel_configs works on scrape targets, metric_relabel_configs operates on the scraped data itself. It lets you drop or modify metrics after they've been collected but before they're sent through the pipeline. This is useful for cleaning up noisy or high-cardinality metrics early.

yaml
123456789
metric_relabel_configs:
# Drop any metric that has the label 'internal_metric'
- source_labels: [internal_metric]
action: drop
# Keep only metrics that match a specific name pattern
- source_labels: [__name__]
regex: "(http_requests_total|rpc_latency_seconds.*)"
action: keep

These two configuration sections give you full control over both what you scrape and how you handle the resulting data. Together, they form the core of most real-world Prometheus receiver setups.

Additional configuration options

The Prometheus receiver includes several other settings and feature gates that let you fine-tune its behavior for more specialized use cases. These options sit at the root of the prometheus receiver configuration, not inside the config block:

  • trim_metric_suffixes: When enabled, this removes suffixes like _total, _sum, _count, and common unit suffixes such as _seconds or _bytes from metric names. This can help standardize metrics across sources, but be careful since changing metric names can affect dashboards, queries, and alerts that depend on them.

  • use_start_time_metric: If set to true, the receiver will look for a metric (by default process_start_time_seconds) and use its value to determine the start time for counters. This can be risky, since it assumes all counters were reset when that process started, which may not always be the case. Only use this option if you know your scraped targets behave predictably in that regard.

yaml
12345678
# otelcol.yaml
receivers:
prometheus:
config:
scrape_configs:
# [...]
trim_metric_suffixes: true # default: false
use_start_time_metric: true # default: false

Scaling scrapes in production with the Target Allocator

Running multiple OpenTelemetry Collectors for high availability introduces a common challenge: by default, every Collector instance scrapes the same set of targets. This leads to duplicate data, unnecessary load on your services, and wasted compute resources.

The Target Allocator (TA) solves this problem. It's the official OpenTelemetry solution for scaling Prometheus scrapes across multiple Collectors in Kubernetes. Instead of having each Collector perform service discovery independently, the TA centralizes that responsibility and ensures each target is scraped by exactly one Collector.

How the Target Allocator works

The Target Allocator changes the way scraping is coordinated by splitting two key functions:

  1. Centralized discovery: The TA handles service discovery centrally. It finds all possible scrape targets—such as annotated Kubernetes pods—and keeps an updated list of them.

  2. Distributed scrapes: Each Collector pod registers itself with the TA. Using a consistent hashing algorithm, the TA divides the list of discovered targets evenly among the active Collectors. This guarantees that each target is scraped once, without overlap.

  3. Dynamic rebalancing: If a Collector pod goes down, the TA automatically redistributes its targets to the remaining pods. When new Collectors are added, it rebalances again to share the load evenly. This happens automatically, without manual reconfiguration.

Behind the scenes, the OpenTelemetry Operator manages the Target Allocator. When enabled, it updates the Prometheus receiver configuration inside each Collector, replacing static discovery blocks with a lightweight http_sd_config. This tells each Collector to fetch its target list from the Target Allocator instead of discovering them itself.

For details on enabling and customizing the Target Allocator, see the official documentation.

How Prometheus data is mapped to OTLP

The Prometheus receiver doesn't just pass data through unchanged. It converts metrics from the Prometheus data model into the richer, structured OTLP format.

Understanding how this mapping works will help you make the most of your data once it reaches the rest of your pipeline:

  • Labels become Attributes: Every Prometheus label is transformed into a key-value attribute on the corresponding OTLP metric data point.

  • Metric types are translated:

    • Prometheus counter → OTLP Sum (cumulative and monotonic)
    • Prometheus gauge → OTLP Gauge
    • Prometheus histogram → OTLP Histogram
    • Prometheus summary → OTLP Summary

Suppose you have this Prometheus counter metric exposed by the Node Exporter:

promql
1
node_cpu_seconds_total{cpu="0", mode="system"} 15342.85

When scraped by the Prometheus receiver, it is converted to an OTLP Sum type with attributes preserved (as seen via the debug exporter):

text
123456789101112131415
Metric #63
Descriptor:
Name: node_cpu_seconds_total
-> Description: Seconds the CPUs spent in each mode.
-> Unit:
-> DataType: Sum
-> IsMonotonic: true
-> AggregationTemporality: Cumulative
NumberDataPoints #0
Data point attributes:
-> cpu: Str(0)
-> mode: Str(system)
StartTimestamp: 2025-10-12 04:39:33.432 +0000 UTC
Timestamp: 2025-10-12 04:44:42.412 +0000 UTC
Value: 15342.85

If the Node Exporter exposes the same metric for multiple CPUs, each combination of labels (for example, cpu="1", mode="user") becomes a separate OTLP data point with its own attributes.

Special mappings for Resource and Scope

The receiver also looks for certain metrics and labels that carry additional context about where the telemetry originated. These are used to populate Resource and Scope attributes in OTLP and enrich your data with metadata about the source and instrumentation.

1. target_info for Resource attributes

If a target exports a metric named target_info (often added automatically through service discovery), its labels are converted into Resource attributes and applied to all other metrics from that same target. After this mapping, the target_info metric itself is dropped.

For instance, a Prometheus metric like:

promql
1
target_info{service_name="auth-api", service_version="1.2.0"}

Will result in all metrics from that scrape including the resource attributes service.name="auth-api" and service.version="1.2.0".

2. otel_scope_info for Scope attributes

If metrics include labels such as otel_scope_name and otel_scope_version, the receiver uses them to build the Instrumentation Scope. This defines which library or component produced the data. The otel_scope_info metric can also provide additional attributes for that scope.

This translation process helps unify Prometheus-style metrics within the broader OpenTelemetry model, giving you structured, context-rich data that's easier to analyze and correlate with other telemetry signals.

Monitoring scrape health with scrape metrics

In addition to the metrics your applications expose, the Prometheus server also automatically generates some metrics that provide visibility into the scraping process itself, including up, scrape_duration_seconds, scrape_samples_scraped, scrape_series_added, and scrape_samples_post_metric_relabeling.

A common question is how the OpenTelemetry Prometheus receiver handles these internal metrics. The good news is that it preserves them exactly as Prometheus does, converting them into OTLP format while maintaining their meaning and labels.

The most important of these is the up metric, a simple gauge that reports 1 when a target was scraped successfully and 0 when the scrape failed. The receiver keeps this behavior intact, exposing up as an OTLP gauge for every target.

Here's what the up metric looks like after being translated into OTLP:

text
123456789101112
Metric #0
Descriptor:
-> Name: up
-> Description: The health of the scraped target.
-> Unit:
-> DataType: Gauge
NumberDataPoints #0
Data point attributes:
-> instance: Str(localhost:9100)
-> job: Str(node-exporter)
Timestamp: 2025-10-12 11:13:27.123 +0000 UTC
Value: 1.000000

With this, you can build the same kinds of availability dashboards and alerts you would in a Prometheus setup, but now within an OpenTelemetry-native environment.

Scraping Prometheus native histograms

Prometheus native histograms are a newer, more efficient way to represent distribution data. They offer better accuracy and performance for high-volume metrics.

The OpenTelemetry Collector Prometheus receiver can ingest these and translate them into OTLP Exponential Histograms, without losing their precision and structure.

To enable this capability, you need to perform two steps: turn on the feature gate in the Collector and configure the scrape protocol to support native histograms.

1. Enable the feature gate

Start by enabling the experimental feature gate when launching the Collector. This allows the Prometheus receiver to process native histograms.

If you're running the Collector with Docker Compose, for example, add the flag under the command section:

yaml
123456789
# docker-compose.yml
services:
otelcol:
image: otel/opentelemetry-collector-contrib:0.137.0
command:
[
"--config=/etc/otelcol-contrib/config.yaml",
"--feature-gates=receiver.prometheusreceiver.EnableNativeHistograms",
]

If you're running it as a Kubernetes Deployment, include the same argument under the container's args field.

2. Configure the scrape protocol

Next, make sure your Prometheus receiver requests a format that supports native histograms. The OpenMetricsText1.0.0 and PrometheusProto protocols are both compatible.

You can explicitly define the protocol order under the global section of the receiver's configuration:

yaml
123456789101112131415
# otelcol.yaml
receivers:
prometheus:
config:
global:
# Preferred scrape protocols that support native histograms
scrape_protocols:
[
OpenMetricsText1.0.0,
PrometheusProto,
OpenMetricsText0.0.1,
PrometheusText0.0.4,
]
scrape_configs:
# ... your scrape jobs here ...

Once both steps are complete, the Collector will correctly negotiate and ingest native histograms so that your metrics retain their accuracy and statistical depth as they flow through the OpenTelemetry pipeline.

Some caveats and unsupported features

While the Prometheus receiver is designed to act as a near drop-in replacement for a Prometheus server, it doesn't implement every feature of Prometheus. Its goal is focused: to scrape metrics and feed them into the OpenTelemetry pipeline. Features related to alerting, long-term storage, or query evaluation remain outside its scope.

If your Prometheus configuration includes any of the following directives, the receiver will return an error on startup and refuse to load the configuration:

  • alert_config.alertmanagers
  • alert_config.relabel_configs
  • remote_read
  • remote_write
  • rule_files
text
123
2025/10/12 11:37:34 collector server run finished with error: invalid configuration: receivers::prometheus::config: unsupported features:
alert_config.alertmanagers
rule_files

These options relate to Prometheus features such as alert delivery, rule evaluation, and data federation—all of which are not part of the Collector's responsibilities. It focuses purely on data ingestion and export, leaving alerting and querying to dedicated backends like Prometheus itself or other metric stores.

If you are migrating an existing Prometheus configuration to the OpenTelemetry Collector, make sure to remove or comment out these sections before loading it into the prometheus receiver. You can still handle equivalent functionality downstream using OpenTelemetry components or external systems:

  • Alerting: Use your metrics backend to define alert rules.
  • Long-term storage: Forward metrics via OTLP or an exporter such as prometheusremotewrite.
  • Recording rules: Implement aggregation or transformation in OpenTelemetry processors instead.

In short, the Prometheus receiver is built for scraping and translating metrics, not for full Prometheus server parity. Keeping your configuration focused on scraping ensures smooth compatibility and fewer surprises during deployment.

Final thoughts

The Prometheus receiver is more than just a way to bring in metrics. It's a full integration point that combines the power of Prometheus with the flexibility of OpenTelemetry. Once you understand how to configure it, you can create a consistent and scalable metric pipeline.

A well-tuned configuration ensures that all your Prometheus data, no matter where it comes from, is standardized, enriched with context, and ready for meaningful analysis.

When your pipeline is stable and clean, the next step is to send your metrics to an OpenTelemetry-native platform like Dash0. Take full control of your observability data and start a free trial today.

Authors
Ayooluwa Isaiah
Ayooluwa Isaiah