Sample Traces

Sampling reduces the volume of traces Dash0 stores while keeping the ones that matter. You define the rules, so you decide which traces to keep. A common setup keeps every error and slow trace and samples routine traces down to a target percentage, but that is one option among many. Spans are held briefly and evaluated together, so a keep-or-drop decision reflects the whole trace rather than any single span. This is tail sampling, and it can reduce trace volume by 90% or more without losing the traces you troubleshoot from.

Sampling applies to spans. Metrics you derive with signal-to-metrics conversion are produced at ingestion, so you keep those numbers even though the stored traces are sampled down. RED metrics are likewise calculated before sampling, so the request, error, and duration views in Dash0 reflect all of your traffic rather than only the sampled traces.

Add a Sampling Rule

1. From the pipeline, open the Sampling stage for Spans and click Add sampling rule.
2. Enter a Display name, for example My Sampling Rule.
3. Under Conditions, choose a Type and fill in its inputs. The condition types are described below.
4. Click Add condition to add another condition, or choose the All of… type to require several at once.
5. Make sure Enabled is on, then click Save.

As you build the rule, the What this rule does panel restates it in plain language so you can confirm the behavior before saving.

Condition Types

The Type dropdown offers four condition types.

Probabilistic

Probabilistic conditions keep a fixed percentage of traces using a deterministic trace-ID hash, so the decision is consistent for every span of a trace. Use them for a baseline sampling rate. Set the Sampling rate to the percentage of traces to keep, between 0 and 100.

Conditional

Conditional conditions keep traces whose spans match a content-based condition. Use them to retain errors, latency outliers, and business-critical flows. Choose how to define the condition with the Editor toggle:

• Filter — build the condition with the same filter controls used elsewhere in Dash0. Click Add filter to add clauses.
• OTTL — write the condition in the OpenTelemetry Transformation Language. Reference span attributes as attributes["..."], resource attributes as resource.attributes["..."], and span fields directly, for example attributes["http.status_code"] >= 500 to keep server-error traces or resource.attributes["k8s.cluster.name"] == "production" to keep traces from the production cluster.

Error

Error conditions keep traces that contain an error. They need no further input.

All of…

All of… conditions require several conditions at once. Nest other condition types inside them, and a trace is kept only when all of the nested conditions match. For example, nest a Conditional rule that matches the production cluster with a Probabilistic rule set to 50% to keep half of your production traces.

A typical rule set combines these: Conditional and Error conditions keep the errors and business-critical traces, and a Probabilistic condition reduces the remaining routine traffic to a baseline percentage.

Sampling on the Edge

When you run SignalControl Edge, tail sampling happens inside your own network, on Kubernetes or without Kubernetes, so the dropped traces never leave your network and you save egress cost as well as storage. Because a single keep-or-drop decision covers a whole trace, ensure all spans of a trace are routed to the same dataset when sampling on the edge.

The rest of this section covers the standalone edge deployment: authoring rules through the API, and the dash0sampling processor and reservoir configuration.

Authoring Sampling Rules via the API

Send API requests to your region's API endpoint from the endpoint table, with an Authorization: Bearer <token> header. Creating sampling rules requires a token with the admin role. See Dash0 API for the general conventions, and the setup steps for a complete curl example.

Rules are created with POST /api/sampling-rules?dataset=<dataset>. A rule has a kind of Dash0Sampling, a metadata.name, and a spec with enabled, an optional display.name shown in the UI, one conditions tree, and an optional rateLimit.

The condition kinds map to the four UI types (ottl is the Conditional type's OTTL editor, and and is All of…):

• probabilistic (Probabilistic): Keep a fixed fraction via a deterministic trace-ID hash. spec.rate is 0.0–1.0. Decided locally in the collector, with no coordination.
• ottl (Conditional): Keep traces matching an OTTL boolean expression in spec.ottl. Reference span fields with span. and resource attributes with resource.attributes[...].
• error (Error): Keep traces with an error span. spec is empty.
• and (All of…): Require all nested conditions. spec.conditions is a non-empty list of the kinds above.

A rule may contain at most 32 ottl and error conditions across its tree. Probabilistic conditions are decided locally and do not count. The API does not reject a larger rule; Dash0 rejects it when loading the rules for your organization, and that rule stops matching. Keep every rule well within the limit. There is no fixed limit on the number of rules per dataset. See the limits table.

Keep every error trace:

12345678910111213141516
{
  "kind": "Dash0Sampling",
  "metadata": {
    "name": "keep-errors"
  },
  "spec": {
    "enabled": true,
    "display": {
      "name": "Keep error traces"
    },
    "conditions": {
      "kind": "error",
      "spec": {}
    }
  }
}

Keep 50% of traces from the production cluster, combining an OTTL condition with a probabilistic one:

12345678910111213141516171819202122232425262728293031
{
  "kind": "Dash0Sampling",
  "metadata": {
    "name": "production-baseline"
  },
  "spec": {
    "enabled": true,
    "display": {
      "name": "Production baseline 50%"
    },
    "conditions": {
      "kind": "and",
      "spec": {
        "conditions": [
          {
            "kind": "ottl",
            "spec": {
              "ottl": "resource.attributes[\"k8s.cluster.name\"] == \"production\""
            }
          },
          {
            "kind": "probabilistic",
            "spec": {
              "rate": 0.5
            }
          }
        ]
      }
    }
  }
}

Keep every slow checkout trace, but no more than 1000 per minute:

123456789101112131415161718192021
{
  "kind": "Dash0Sampling",
  "metadata": {
    "name": "slow-checkout"
  },
  "spec": {
    "enabled": true,
    "display": {
      "name": "Slow checkout traces"
    },
    "conditions": {
      "kind": "ottl",
      "spec": {
        "ottl": "span.name == \"POST /checkout\" and span.duration > 500ms"
      }
    },
    "rateLimit": {
      "rate": 1000
    }
  }
}

rateLimit is optional but recommended on any rule that can match a burst. rateLimit.rate caps traces kept per minute for the rule, so an outage that trips an error rule does not flood ingest. Very low limits (16 or below) are not guaranteed, and a probabilistic-only rule cannot set one.

Sampling Processor Configuration

Every field on dash0sampling:

The Reservoir

The reservoir buffers spans while their trace waits for a decision. Pick a type by retention need and throughput:

disk is what the processor uses when type is unset, and it refuses to start without a data_dir. The reference configuration sets serialized_memory explicitly, which needs no storage and suits most workloads. Choose disk deliberately when you need long retention or volumes that exceed memory, and give it a dedicated fast volume.

Reservoir fields:

Sizing rule of thumb: required storage ≈ spans/sec × bytes/span × buffer_duration, across the fleet. At 800 bytes per span, 100k spans/s over a 10s window is about 800 MB; split it across your collector instances.

Further Reading

• About SignalControl. Overview of the full reduction pipeline and how the stages compose.

• About SignalControl Edge. Run tail sampling inside your own network, on Kubernetes or standalone, to also cut egress cost.

• About Tracing. How Dash0 ingests, stores, and explores traces and spans.

• Convert Signals to Metrics. Derive metrics at ingestion so they survive trace sampling.