Skip to content
Benchmaker

Glossary · Objects and system context

Measurement version

5Lifecycle

The unit of evaluator versioning: rubric text, examples, judge model and prompt, checkers, thresholds, aggregation logic, and expert policy, versioned together. Changing a load-bearing part creates a new measurement form: revalidate it or run a bridging study.

Defined in

Handbook · LifecycleHarness architecture and reproducibilityThe harness is part of the instrument. Isolate trials, pin environments, version judges, and log enough to turn a surprise into a result.
Eval harness
The machinery that administers an evaluation and records trials. It is part of the instrument, distinct from the agent harness, which is part of the system under test.
Versioned dynamism
Dynamic corpus, static run: refresh an eval suite deliberately across versions, but freeze every environment snapshot, tool contract, scorer, and dependency needed to reproduce a reported result. A new snapshot is a new measurement form.

Where it comes up

See it in the full glossary →