Dash0 acquires Polar Signals

Last updated:

Manage Evals

Enable and disable evals, edit their settings, review previous versions, and delete evals you no longer need.
Beta

Evals is in beta. Eval runs are not charged during the beta. Behavior and UI may change.

To change an eval, you need the Maintain permission on its dataset, granted to your role, a team, or you individually. Without it, the eval opens read-only and shows a Read-only tag. See Access Control.

Open AI Observability → Evals and click an eval to manage it.

Eval header with the eval name, version selector, Enabled toggle, and delete icon above the Overview and Settings tabs

Enable or Disable an Eval

Use the Enabled toggle in the eval's header. The change saves immediately, separately from any unsaved edits on the Settings tab.

A disabled eval stops judging new spans. Its existing results stay available. When you enable it again, it judges spans that arrive from then on. It does not judge spans that arrived while it was disabled.

Edit an Eval

  1. Open the Settings tab.
  2. Change the settings you need. See Create an Eval for each step.
  3. Click Save to store the changes as a new version, or Discard to drop them.

You cannot change an eval's output type after it is created. To use a different output type, create a new eval.

Changes apply to spans that arrive after you save. Existing verdicts keep the version that produced them.

Note

If someone else saves the same eval while you are editing it, Dash0 rejects your save with the message This eval was changed by someone else while you were editing. Reload it and reapply your changes. Reload the page and make your changes again.

Review Previous Versions

Every saved change creates a new version. Click the version number next to the eval's name, for example (v2), to open a previous version. A previous version shows a Previous version tag and opens read-only.

Selecting Open eval definition on a verdict in the span sidebar opens the version that produced that verdict.

Delete an Eval

  1. Click the delete icon in the eval's header.
  2. Confirm in the Delete this eval? dialog.

Deleting an eval cannot be undone. The eval stops judging spans and no longer appears in the evals list.

Further Reading

  • Analyze Eval Results — Read scores, outcomes, and labels per eval and per span.
  • Create an Eval — Define which spans to judge, the judge model, and the output.
  • Key Concepts — Terminology for triggers, sampling, verdicts, and credits.
  • Write Eval Prompts — Reference span attributes and ask for the right output shape.