Skip to content
Benchmaker

Glossary · Use and training pipelines

Release gate

3Regime

An evaluator that determines whether a system or version can ship: a score-to-decision mapping with a threshold. This handbook treats a pre-release gate as measure until developers iterate against it; a request-path threshold is runtime control.

Defined in

Handbook · RegimeUse regimes: same judge, four jobsMeasure, optimize or monitor: an evaluator changes meaning when its signal enters a different loop, and signals migrate between loops.
Use regime
The causal role a signal plays: measure (supports inference), optimize (shapes the candidate or policy distribution), or monitor-control (detects and intervenes in deployment).
Regime migration
When a signal's causal role changes, such as a benchmark becoming an optimization target or a monitor becoming a training signal. Its validation obligations move with it.
Runtime control
An evaluator that can block, reroute, escalate, or roll back deployed behavior, rather than only observe it.

Where it comes up

See it in the full glossary →