AutoMerge is generally available

Last updated:

dash0_eval

Terraform resource for Dash0 generative AI evaluations (evals) that score GenAI spans with an LLM judge.

Manages a Dash0 generative AI evaluation (eval). An eval sends a sample of the GenAI spans that match its filters to an LLM judge, which scores each span with the eval's prompts.

Note

metadata.name and the kind of spec.judge.output cannot change after create, because eval results are stored under the name and per output kind. Changing either destroys the eval and creates a new one, and the new eval starts with no results.

metadata.name must be unique within the dataset, and spec.match.filters must contain an is_set filter on gen_ai.operation.name without values. Deletes are soft: a destroyed eval stops judging spans, and the API keeps a tombstone of it. Changing dataset destroys the eval and re-creates it under a new origin.

Example Usage

terraform
resource "dash0_eval" "answer_relevance" {
  dataset   = "default"
  eval_yaml = file("${path.module}/eval.yaml")
}

Schema

Required

  • eval_yaml (String) The eval definition in YAML format. The YAML must declare kind: Dash0GenerativeAiEvaluation, metadata.name, an explicit spec.enabled, and spec.display, spec.match, spec.sample and spec.judge. The dash0.com/folder-path and dash0.com/sharing metadata annotations are supported, and changes to them trigger a resource update; dash0.com/sharing has no effect on access. The server-managed metadata.labels and metadata.annotations are ignored, and any other label or annotation is rejected, because the Dash0 API would drop it.

Optional

  • dataset (String) The identifier of the Dash0 dataset that the eval belongs to. Provide the dataset's identifier, which is immutable, not the 'name'. If omitted, the provider-level dataset default is used (see the provider's dataset attribute). Changing this value forces the resource to be recreated under a new origin.

Read-Only

  • id (String) The server-assigned identifier of the eval, resolved by the provider after creation.
  • origin (String) A unique identifier for the eval, automatically generated on creation. Used to reference the eval for updates, reads, deletes, and imports.

Import

Import is supported using the following syntax:

The terraform import command can be used, for example:

shell
#!/bin/bash
terraform import dash0_eval.answer_relevance default,tf_existing-evaluation-origin