Last updated:
dash0_eval
Manages a Dash0 generative AI evaluation (eval). An eval sends a sample of the GenAI spans that match its filters to an LLM judge, which scores each span with the eval's prompts.
metadata.name and the kind of spec.judge.output cannot change after create, because eval results are stored under the name and per output kind. Changing either destroys the eval and creates a new one, and the new eval starts with no results.
metadata.name must be unique within the dataset, and spec.match.filters must contain an is_set filter on gen_ai.operation.name without values. Deletes are soft: a destroyed eval stops judging spans, and the API keeps a tombstone of it. Changing dataset destroys the eval and re-creates it under a new origin.
Example Usage
resource "dash0_eval" "answer_relevance" {
dataset = "default"
eval_yaml = file("${path.module}/eval.yaml")
}Schema
Required
eval_yaml(String) The eval definition in YAML format. The YAML must declarekind: Dash0GenerativeAiEvaluation,metadata.name, an explicitspec.enabled, andspec.display,spec.match,spec.sampleandspec.judge. Thedash0.com/folder-pathanddash0.com/sharingmetadata annotations are supported, and changes to them trigger a resource update;dash0.com/sharinghas no effect on access. The server-managedmetadata.labelsandmetadata.annotationsare ignored, and any other label or annotation is rejected, because the Dash0 API would drop it.
Optional
dataset(String) The identifier of the Dash0 dataset that the eval belongs to. Provide the dataset's identifier, which is immutable, not the 'name'. If omitted, the provider-leveldatasetdefault is used (see the provider'sdatasetattribute). Changing this value forces the resource to be recreated under a new origin.
Read-Only
id(String) The server-assigned identifier of the eval, resolved by the provider after creation.origin(String) A unique identifier for the eval, automatically generated on creation. Used to reference the eval for updates, reads, deletes, and imports.
Import
Import is supported using the following syntax:
The terraform import command can be used, for example:
#!/bin/bash
terraform import dash0_eval.answer_relevance default,tf_existing-evaluation-origin