This guide explains how Dash0's log storage works, why certain filters are faster than others, and practical strategies to improve query performance, ordered from most to least impactful.

How Log Storage Works

Storage Layout and Partition Pruning

Dash0 stores log records in a columnar storage engine partitioned by time and resource attributes. When a query includes a resource attribute filter (e.g., service.name), the engine skips entire partitions that cannot contain matching data. This is called partition pruning and is the single biggest factor in query speed.

Column Statistics and Attribute Selectivity

Each data partition maintains column-level statistics such as min/max values and null counts. These statistics allow the engine to skip partitions even for non-resource attributes, but only when the filter value falls outside a partition's recorded range.

This means an attribute filter is only useful for pruning if the attribute values are selective. If an attribute key exists on every log record with near-uniform distribution, column statistics cannot eliminate any partitions and the engine must scan all of them. Understanding this distinction is key to choosing effective filters.

Optimization Strategies

1. Filter by Resource Attributes First

This is the single most effective optimization. Resource attributes determine how data is partitioned, so filtering on them prunes entire swaths of data before any log record is read.

1
service.name is checkout-service

The most effective resource attributes for filtering:

service.name : Almost always your first filter.

k8s.namespace.name : Kubernetes namespace isolation.

k8s.deployment.name : Workload-level filtering.

deployment.environment.name : Separate production from staging.

dash0.resource.name : Works for all resource types.

In the Log Explorer, these attributes are visible in the Resource column and available in the filter bar.

2. Start with Adaptive Sampling

When browsing logs without a specific hypothesis, Adaptive Sampling keeps the UI responsive regardless of data volume by intelligently sampling during query execution. This lets you scan large datasets quickly to spot patterns before committing to a focused query.

Once you identify an interesting pattern or anomaly, add filters to narrow the results. With selective filters in place, switch to Precision mode for full visibility over the filtered dataset.

Tip

Adaptive Sampling is especially useful during incident investigations when you do not yet know which service or attribute to focus on.

3. Use Severity Filters

Severity is stored as a dedicated column with good partition statistics. Filtering to a specific severity range allows the engine to skip partitions that contain only lower-severity records.

1
otel.log.severity.range is ERROR

Combining a severity filter with a resource attribute filter compounds the pruning effect. See Identify Log Severities for an overview of severity ranges.

4. Filter on Log Attributes Selectively

Log attributes (non-resource attributes on individual log records) can benefit from partition pruning via column statistics, but their effectiveness depends on selectivity.

Scenario	Example filter	Pruning benefit
Attribute exists on a subset of records	`http.status_code is 500`	High: partitions without this attribute are skipped
Attribute has clustered values	`k8s.container.name is api-server`	Moderate: partitions with different containers are skipped
Attribute exists on every record with unique values	`request.id is abc-123`	Low: every partition contains the attribute, statistics cannot prune

Prefer attributes with low cardinality and clustered distribution. Attributes derived from resource context (deployment name, container name) tend to work better than per-request identifiers.

5. Use Full-Text Search on `otel.log.body`

The otel.log.body attribute has a full-text search index. Using the contains operator on it locates matching records through the index without scanning every log body:

1
otel.log.body contains "connection refused"

This is a qualitatively different, and much faster, operation compared to contains on other attributes, which must perform a brute-force scan across all records in scope.

Attribute	`contains` performance	Reason
`otel.log.body`	Fast	Full-text search index
`dash0.log.message`	Slower	No full-text index; brute-force scan
Other string attributes	Slower	No full-text index; brute-force scan

Note

Despite having similar content, dash0.log.message does not benefit from the same full-text index as otel.log.body. Always prefer otel.log.body when searching log message text.

6. Choose Appropriate Time Ranges

Narrowing the query window directly reduces the volume of data scanned:

Investigations : Start with a narrow window (e.g., "Last 30 minutes") and widen only if needed. Use the severity chart's click-and-drag time selection to zoom into a specific spike.

General browsing : Data within the last 24 hours is served from fast local storage. Queries reaching beyond 24 hours retrieve data from object storage, which adds latency.

Combining Strategies

These strategies compound. The fastest log queries combine a resource attribute filter with a severity filter and a narrow time range. Start with Adaptive Sampling for exploration, then layer on filters as you narrow the investigation. A single well-chosen resource attribute filter often has more impact than all other strategies combined.

Summary

The most impactful optimizations are filtering by resource attributes (especially service.name), using Adaptive Sampling for initial exploration, and applying severity filters to prune irrelevant records. For text searches, always use the contains operator on otel.log.body to leverage the full-text index. Keep time ranges narrow and prefer selective, low-cardinality attribute filters.

For a full reference on the filtering UI, see Browse and Filter Logs.

Optimize Log Query Performance