Skip to content
Benchmaker

Glossary · Optimization and failure dynamics

Judge rot

5Lifecycle

The judge's prompt, model, or settings change without versioning and recalibration, and every downstream number inherits the error. Version the whole measurement and revalidate on change.

Defined in

Handbook · PracticeAnti-patterns and failure vocabularySeventeen named ways evaluator systems fail, each with its tell, its layer and its fix.

See it in the full glossary →