Monitor your agents' quality with Evals
Evals score your agents' generations as they arrive, with an LLM as the judge, so quality sits next to latency and cost. Write your own eval, then pick a trigger, a sample rate, and a numeric, boolean, or categorical verdict.
Until now, Dash0 could tell you how long each of your agents' answers took and what it cost, but not whether it was any good. Finding out meant exporting traces into a separate eval tool, where the scores never lined up with the rest of your telemetry.
Evals score your agents' generations in Dash0 as the spans arrive. You choose which spans to judge with a trigger and span filters, set a sample rate to control cost, and write the prompt the judge follows. The judge returns a numeric score, a true or false answer, or a label from a list you define, with a one-sentence explanation for every verdict.
Each eval has its own overview, with scores over time, the lowest-scoring spans, and the credits spent per agent. Open any judged span and its Evals tab shows every verdict right next to the trace. Verdicts are also stored as log records, so you can chart them on dashboards and alert on them with check rules.
Evals are in beta, and eval runs are free during the beta. Read the documentation to create your first eval.