Dash0 updates from March 2025

​We are excited to introduce the new In-Product Home Page in Dash0, designed to provide a high-level overview of the telemetry sent to your account. This centralized dashboard offers key insights into your system's performance and health at a glance.​

Key Features

• Comprehensive Overview: Gain immediate visibility into critical telemetry data, including metrics, logs, and traces, all in one place.​

• User-Friendly Interface: The intuitive design ensures that both new and experienced users can navigate and interpret the data effortlessly.​

This enhancement aims to streamline your observability experience by consolidating essential telemetry information into a single, accessible location. We believe this will empower you to make more informed decisions and respond to system changes more swiftly.​

Dash0 now supports real-time notifications via PagerDuty when a checkrule fails. Incidents are automatically closed once the checkrule returns to a healthy state, streamlining incident management and reducing manual effort.

Key Features of the PagerDuty Integration

• **Real-time PagerDuty Notifications on Checkrule Failures: **Trigger immediate alerts to PagerDuty when a Dash0 checkrule enters a failure state. Ensure incidents are assigned with appropriate urgency and context.

• **Automatic Incident Resolution on Checkrule Recovery: **Seamlessly resolve open PagerDuty incidents when the associated checkrule returns to a healthy state, minimizing alert fatigue and manual intervention.

• **Enhanced Annotation Display with Hyperlinked References: **Render any URLs within checkrule annotations as clickable hyperlinks improving visibility and quick access to supporting dashboards, documentation or tools.

More details on how to set up the PagerDuty integration can be found in our integration hub.

The new Dash0 Linkerd Integration combines service mesh metrics and trace data into a powerful, ready-to-use dashboard giving you:

• RED Metrics (Rate, Errors, Duration) for all your Linkerd services

• Linkerd Proxy resource usage across the mesh

• An overview of all pods participating in the mesh

• A detailed breakdown of operations handled by Linkerd data plane proxies

All you need is an existing Linkerd deployment, the OpenTelemetry Collector, and your Dash0 token - check the Integration Hub to get started.

Our new Slack Bot integration, found at the integration hub, now includes an image of the latest chart with every alert. This image updates with each status change, giving you a clear visual representation of what happened during the alert's time range and shortly before.

Key Features

• Better overview: Gain a clearer understanding of the failed check within the alert's time frame.

• History: View a history of all chart images within the alert thread.

• Real-time updates: The chart image in the main message updates with every status change.

This new feature aims to provide a more intuitive and informative experience, allowing you to quickly grasp the situation and stay on top of your alerts.

It was a good idea, but we did not execute it right: we saw users struggle with "going back” to larger timeframes, and feeling stuck. Internally, we “affectionately” called this user-experience issue "lost in time."

We went back to the drawing board, and think we found a nice solution: Now, the lists of spans and logs in the Logging and Tracing views filter themselves as soon as a selection is made. You can drill down in a subset of the raw telemetry, without losing the global context. No additional clicks, no "lost in time". Yet, you still have the option to zoom if you want to!

We recently announced the availability of custom filter rules in the operator as well as a similar filter mechanism in our platform, called Spam filters, that lets you discard useless telemetry in the Dash0 backend.

Today these capabilities are complemented by a new operator feature that allows you to transform telemetry collected by the operator with custom rules before it is sent to Dash0. In this context, filtering is about dropping entire objects (spans, log records, metrics) according to specific rules, but transforming telemetry goes one step further and gives you even more flexibility and power. Here are some things that you can do with telemetry transformations:

• Remove span attributes without discarding the span as a whole (or metric attributes, log record attributes, …).

• Limit the number of attributes on objects such as spans or metric data points.

• Limit the length of attributes.

• Modify resource attributes.

• Add attributes, based on the value of other attributes and conditions.

• Change the value of attributes (think regex replacements).

• Change the type of metrics (e.g. turn a sum into a gauge)

• …and probably a gazillion other things.

As usual, the Dash0 operator allows you to configure these rules per Kubernetes namespace. Here is an example of a Dash0 monitoring resource that configures a set of transformations:

dash0-monitoring.yamlapiVersion: operator.dash0.com/v1alpha1
kind: Dash0Monitoring
metadata:
  name: dash0-monitoring-resource

spec:
 
  transform:
    trace_statements:

      # limit span attributes generally to 4096 characters
      - 'truncate_all(span.attributes, 4096)'

      # remove sensitive data from collected command line invocations 
      - replace_pattern(span.attributes["process.command_line"], "password\\=[^\\s]*(\\s?)", "password=***")

      # remove query parameters (using the advanced config style here)
      - conditions:
        - 'span.attributes["http.target"] != nil'
        statements:
        - 'replace_pattern(span.attributes["http.target"], "customerId=.*", "customerId=xxx")'

    log_statements:
     
      # change the log level based on conditions
      - conditions:
        - 'log.body == "request failed"'
        statements:
        - 'set(log.severity_text, "FAIL")'

The configuration syntax is deliberately chosen to be identical to what the OpenTelemetry transform processor allows, supporting both the basic flat statement style as well as the advanced configuration style that supports adding conditions to a single transformation. This allows you to copy and paste existing transformation rules that you already have, or use examples you find online as-is.

For more details on telemetry transformations, check out the operator documentation.

Analysis methods

Besides comparing a subset of spans with all the rest in the global timeframe, it gives you ways to:

• Analyze attribute values: analyze how attributes and their values cluster across all the available spans. Maybe you are not looking to troubleshoot issues, but rather understand the reality of your data.

• Compare with the other spans in the same timeframe: this method is very good to analyze outliers, especially in terms of duration.

• Compare with everything *before *the selection in the global timeframe: this is an excellent way to analyze the first occurrence of spikes or error clusters.

Triage is in Beta

This is the first iteration of Dash0 triage. It is available in open beta for Tracing, which is the signal with the largest amount of metadata on average.

At time of writing this blog, we know of various cosmetic bugs and UX details that need improvement, but we don’t want to wait to learn with and from you and your use cases. Also, we are working hard on the performance (sifting on a lot of data the way Triage does requires optimizations), and until we are happy with the outcome, you can use Triage only on timeframes of up to 24 hours.

Triage will also come for Logging and Metrics when we feel we have confirmation through your feedback that we nailed user experience. Indeed, expect the user experience to be tuned and improved in the next few days and weeks.

To help you curb unnecessary telemetry, we have introduced Spam Filters. With two clicks, you can define filtering rules, executed by the Dash0 ingestion pipeline, to drop logs, metric data points and spans before Dash0 charges you for it.

(You read it correctly: there is no extra charge for spam filter rules, and data that is dropped by them, does not count against your Dash0 bill.)

Any combination of filters you can specify in the Logging, Tracing or Metrics pages can be synthesized as Spam filters. You can also export those filters as OpenTelemetry Collector configurations using the filter processor and OTTL (which is pretty much how we implemented it in our ingestion pipeline), so that you can deploy the same filters in your infrastructure and be doubly sure.

The Spam filters are listed in your dataset configurations, and different datasets can specify different filters.

After you create a Spam filter, matching telemetry starts getting dropped in a few seconds, but you might get a few last “stragglers” in. Similarly, after you remove a Spam filter, it will take a few seconds for the change to take effect. Telemetry that has been dropped by a Spam filter is not recoverable by Dash0.

In a later update, we will add tracking for just how much data you dropped. (You know, for bragging rights and so.)

Oh, by the way, we added similar functionality to drop telemetry in the Dash0 operator.

Note about spans: Spam filters drop single spans, you might end up with incomplete traces (unlike what you do with client-side sampling).

In the same vein as the Spam filters you find in Dash0, we exposed the possibility of configuring the OpenTelemetry Collector filter processor via the Dash0Monitoring resource in the Dash0 operator.

With the new filter capabilities of the Dash0 operator, you can discard some telemetry before it even leaves your cluster:

dash0-monitoring.yamlapiVersion: operator.dash0.com/v1alpha1
kind: Dash0Monitoring
metadata:
  name: dash0-monitoring-resource
spec:
  filter:
    traces:
      span:
      - 'attributes["http.route"] == "/ready"'
      - 'attributes["http.route"] == "/metrics"'
      spanevent:
      - 'attributes["grpc"] == true'
      - 'IsMatch(name, ".*grpc.*")'
    metrics:
      metric:
      - 'name == "k8s.replicaset.available"'
      - 'name == "k8s.replicaset.desired"'
      datapoint:
      - 'metric.type == METRIC_DATA_TYPE_SUMMARY'
      - 'resource.attributes["service.name"] == "my_service_name"'
    logs:
      log_records:
      - 'IsMatch(body, ".*password.*")'
      - 'severity_number < SEVERITY_NUMBER_WARN'

The configurations above will apply only to telemetry coming from the namespace in which this Dash0Monitoring resource resides.

You can also export the OpenTelemetry Transformation Language (OTTL) expressions from your Spam filters in Dash0, and apply it to the relevant namespaces in a “belt and suspenders” kind of fashion.

Having the option to filter telemetry both at the Dash0 backend as well as directly in your Kubernetes cluster might seem redundant, but it is actually useful for a couple of reasons:

• Using the spam filter in Dash0 is a quick and easy way to remove telemetry that has little or no value, and to reduce your Dash0 bill.

• However, if some components in your cluster really produce a lot of noisy telemetry, you would still pay for egress as the data is sent to Dash0, only to have Dash0 discard it immediately.

• Filtering directly in your cluster enables you to optimize your cloud costs further by removing those egress costs as well.

• Filtering in your cluster can also help remove sensitive information before it leaves your cluster – the log record filter above which drops lines with “password” is an example for that. (Ideally, telemetry with sensitive information wouldn’t even be produced at all, but mistakes happen.)

Note: You should be able to reuse the configurations you may already have (or find on the Internet or generate via AI) for the filter processor of the OpenTelemetry Collector.

• Effortless Aggregation: Now, you can easily group your logs and spans based on any attribute, allowing you to quickly identify patterns and anomalies.

• Contextual Insights: For each group, you'll get clear statistics to guide your workflow.

• Seamless Drill-Down: Need to investigate a specific group further? Simply click to drill down and explore the detailed logs or spans within that group.

But sometimes, you want to focus just on one aspect, like error percentage or duration percentiles. You already could today in many visualizations of Dash0, like the Services list and tables, but now we brought it to the Tracing view as well in the shape of the RED Charts.

**The RED Charts are an alternative visualization **alongside our Tracing Heatmap within the Tracing Explorer. This powerful combination balances detailed analysis with actionable clarity:

• Heatmaps reveal nuanced latency patterns and request distributions over time

• RED Charts highlight systemic trends in request volume, errors, and durations.