Collect the CloudWatch metrics you need, skip the ones you don't

Until now, getting CloudWatch metrics into Dash0 meant running a pipeline in your own AWS account: a CloudWatch Metric Stream in every region, a Firehose delivery stream behind it, and an S3 bucket for failed deliveries. It is fast and complete, and for many teams that is the right trade-off. It also sends every series CloudWatch publishes, so your AWS bill and your Dash0 bill grow with all of them.

Today we are adding a second way: CloudWatch metrics scraping in the Dash0 AWS integration. You give Dash0 read-only access to your account. Dash0 finds the metrics that exist, picks the ones worth collecting, and scrapes them every five minutes. You decide what gets in with the AWS tags you already use. Metric Streams stay fully supported, so you can pick the trade-off that fits each account.

What you get

• No pipeline to run: no Metric Streams, no Firehose, no backup buckets. Dash0 uses a read-only IAM role that the CloudFormation template sets up. → Set up the AWS integration
• Only the detail you need: CloudWatch publishes many metrics several times, once per dimension combination. Dash0 keeps the most detailed version and skips the totals that only repeat it. → Collect AWS CloudWatch Metrics
• Scrape by tag: choose what to collect with AWS resource tags and regions, for one namespace or for the whole integration. → Filter AWS Metrics by Resource Tags
• Your tags in Dash0: the AWS tags of your resources become attributes in Dash0, so you can filter metrics, logs and traces by team, env or any tag you already have. → Use AWS Resource Tags in Dash0
• Same metric names: scraped metrics use the same names as metrics from a Metric Stream, for example aws.lambda.invocations.

How Metric Streams work

A CloudWatch Metric Stream pushes metrics out of CloudWatch as soon as AWS records them. You create one stream per account and region, and it delivers through Amazon Data Firehose to Dash0.

You choose what a stream sends in two ways:

• Namespace: for example AWS/Lambda.
• Metric name: for example only Invocations.

That is as fine as it gets. You cannot pick a dimension combination, a resource, or a tag.

This matters because CloudWatch publishes the same metric several times. Take Invocations for a Lambda function. CloudWatch publishes it per function, again per function and alias, and again per function, alias and version. On top of that comes one total for all functions in the account. The coarser numbers are just sums of the most detailed one. A stream sends all of them. Multiply that by every metric, every function and every region, and a large share of what arrives repeats something you already have.

Each of those copies is charged:

• On AWS: Metric Streams are billed per metric update, and Firehose per gigabyte it delivers.
• In Dash0: every data point that arrives is ingested.

With streaming, the way to trim this is a spam filter in Dash0. It drops data on arrival, which lowers the Dash0 bill. The AWS charges stay, because by then the data has already been streamed.

How scraping works

Scraping turns the flow around. Instead of AWS pushing everything, Dash0 asks for exactly what it needs. Every five minutes, for each namespace you turned on, Dash0 runs through these steps:

1. Find what exists. Dash0 lists the metrics that are actually published in your account and region, with their dimension combinations.
2. Pick the most detailed version. When a metric exists for the same resource in several dimension combinations, Dash0 keeps the most detailed one and drops the ones that only repeat its total. For the Lambda function, Dash0 keeps Invocations per function, alias and version, and drops the per-alias, per-function and account-wide totals. You can still get any of those totals in a query by summing.
3. Check your filters. Dash0 matches each metric's resource against your conditions, for example team = payments or a list of regions. Resources that don't match are dropped. When you set a tag condition, resources without that tag are dropped too.
4. Fetch only what is left. Dash0 requests datapoints only for the metrics that passed steps 2 and 3. Everything dropped before this step causes no CloudWatch data request and no data in Dash0.
5. Attach identity and tags. Each metric is linked to its resource through the resource's ARN, and the resource carries its AWS tags. That is what lets you filter by tag later.

The result: you control what reaches Dash0 with a few conditions in the Metrics tab, and your AWS bill shrinks with it, because the unneeded data is never requested in the first place. The trade-off is time: scraped data arrives every five minutes, so when your alerts need data within seconds, Metric Streams remain the better fit.