Start an eval from a template
Pick one of 28 eval templates instead of writing an eval from scratch. Each template comes with span filters, a sample rate, a judge model, and a judge prompt, and you can change any of it before you save.
A blank eval form asks a lot. You have to know which spans to judge, how many to sample, which model should judge them, and how to write a rubric it applies the same way every time. Most teams know what worries them, like hallucinations or users getting fed up, but not how to turn that into a prompt.
Eval templates give you the prompt. Click Create new on the Evals page to open the catalog, pick a template, and the eval editor opens with everything filled in.
What's in the catalog. 28 templates in five categories, from Hallucination and Tool selection to User frustration and PII leakage. Top picks gives you a place to start.
What a template fills in. Everything the eval form asks for: span filters, a 20% sample rate, the judge model and prompts, and the output type. Each system prompt spells out what counts and what doesn't. The Hallucination judge, for example, doesn't count a clarifying question or a faithful summary of the context as a hallucination. Change whatever doesn't fit your agents, then save.
What you end up with. An ordinary eval. Edit, disable, or delete it like one you wrote yourself. A later change to the template never changes your eval, so its scores stay comparable over time.
Read Eval Templates to see every template and how to start from one.