Skip to content
Benchmaker

Glossary · Meaning and specification

Evidence contract

2Instrument

The part of a criterion card that states which observable evidence is admissible for that criterion: for example cited spans for grounding, or the tool trace and state transition, never the final prose, for a forbidden action.

Defined in

Handbook · InstrumentAnchors and evidence viewsWhat the judgment is rooted in, and what the evaluator may see: two coordinates most taxonomies forget, and the cheapest lever for better evaluators.
Evidence view
What the evaluator may observe: the final answer only, or also sources, tool traces, hidden state, snapshots, provenance and logs.
Criterion card
The atomic specification for one criterion: facet, operational definition, allowed evidence, positive and negative exemplars, boundary and abstention rules, severity (fatal, tradeable, advisory, or escalate), mechanism, blind spots, owner, and version.

Where it comes up

See it in the full glossary →