Skip to content
Benchmaker

Glossary · Optimization and failure dynamics

Versioned dynamism

5Lifecycle

Dynamic corpus, static run: refresh an eval suite deliberately across versions, but freeze every environment snapshot, tool contract, scorer, and dependency needed to reproduce a reported result. A new snapshot is a new measurement form.

Defined in

Handbook · LifecycleLiving eval suitesThe asset is the machinery that notices when a score stops meaning what it used to: tripwires, drift and saturation.
Linking items
Tasks carried unchanged across versions of a refreshed eval suite, with overlap samples, so that scores on different versions can be linked. Some sources call them anchor tasks; this handbook reserves 'anchor' for reference anchors and the anchor channel.
Drift
Change in task, environment, user, generator, or evaluator distribution that invalidates earlier calibration or validity evidence.

Where it comes up

See it in the full glossary →