Glossary · Evaluator mechanisms
Judge
A learned or prompted evaluator without formal soundness guarantees. People who score are raters, panels, or adjudicators, not judges, in this handbook.
Defined in
Handbook · FoundationsVocabulary without category errorsReward model, rubric, PRM, RLHF, verifier: put each term on the axis it actually describes and most confusions disappear.Related terms
- Verifier
- Strictly, an evaluator with a soundness guarantee relative to a formal specification. In loose usage it often means a deterministic or execution checker.
- LLM-as-a-judge
- A prompted general-purpose LLM evaluating another artifact, usually with a rubric, references, or pairwise comparison.
Where it comes up
- Decision claimsHandbook · Claim
- LLM judges in practiceHandbook · Instrument
- The judge as a noisy sensorHandbook · Validation