Key Concepts

An eval decides which spans to judge, how often, with which model, and what shape of answer to expect. Every term below maps to a field on the eval's Settings tab or to part of the result it produces.

Terminology

The terminology for Evals, alphabetically ordered, is as follows.

Credits

The cost of running an eval, counted in Agent0 credits and derived from the judge model's token usage. The Credits column on the evals list and the Breakdown card on an eval's Overview tab show credits consumed. Eval runs are not charged during the beta. See Model Selection for how Agent0 credits relate to model classes.

Eval

A rule that scores GenAI spans with an LLM judge. An eval belongs to one dataset and only judges spans in that dataset. It combines a trigger, span filters, a sample rate, a model class, an output type, and two prompts. See About Evals for more information.

Explanation

The one-sentence justification the judge returns with every verdict. The span sidebar shows the explanation for failing verdicts. Every result record carries it in gen_ai.evaluation.explanation.

Intent

The polarity you assign to each label of a categorical eval: Positive, Neutral, or Negative. Intent does not make a verdict pass or fail. Labels default to Neutral.

Judge

The LLM that reads the prompts for a sampled span and returns a verdict. You choose its model class, not a specific model.

Model Class

The class of judge model an eval uses:

• Light: Fast, low-cost evaluations. This is the default.
• Standard: Balanced quality and cost.

Output Type

The shape of answer the judge returns. You cannot change it after the eval is created.

• Numeric: A score from 0.0 to 1.0. Higher scores are always better.
• Boolean: True or false. True is the passing answer.
• Categorical: One label from a list you define. Each label has an intent.

Pass and Fail

Whether a verdict counts as passing depends on the output type:

• Numeric: Passes when the score is at or above the threshold. Without a threshold, a numeric verdict always passes.
• Boolean: Passes when the value is true.
• Categorical: Has no pass or fail outcome.

Sample Rate

The percentage of matching traces an eval judges. Sampling is deterministic by trace ID: a trace is either sampled for all of its matching spans or for none of them. New evals default to 100%.

Span Filters

The conditions a span must meet to be judged. Every eval includes the locked filter gen_ai.operation.name is set, plus any filters its trigger adds and any filters you add. All filters must match. Filters resolve against span attributes first, then resource attributes.

Threshold

The optional score at which a numeric verdict counts as passing, between 0.0 and 1.0.

Trigger

A preset that adds span filters for a common kind of GenAI span:

When you add your own span filters, the trigger shows as customized, for example Tool executed (customized).

Verdict

The result of judging one span with one eval: a score, a true or false value, or a label, together with an explanation. Dash0 stores each verdict as a log record linked to the judged span. See Eval Result Records.

Version

Each saved change to an eval creates a new version. A verdict records the version that produced it, so you can open the exact definition behind any result. Enabling or disabling an eval saves immediately, separately from your other changes.

Further Reading

• About Evals — What evals do, where to find them, and their limitations.
• Create an Eval — Define which spans to judge, the judge model, and the output.
• Eval Result Records — Attributes on each stored verdict.
• Write Eval Prompts — Reference span attributes and ask for the right output shape.