Skip to content
Benchmaker

Glossary · Evaluator mechanisms

Reliability–coverage trade-off

Signal reliability rises toward the top of the verification hierarchy while task coverage widens toward the bottom. The response is to decompose the judgment and push each criterion as high as it can legitimately go.

Also called: reliability–coverage frontier

Defined in

Handbook · FoundationsThe verification hierarchy, demotedFormal > execution > learned judges > intrinsic signals. A useful prior on how grounded a signal is, never a verdict on validity.
Verification hierarchy
Formal verification > execution feedback > learned judges > intrinsic signals: a prior over expected grounding strength, not a verdict on validity. Human judgment sits outside the ladder as the external frontier and trust root.

See it in the full glossary →