Skip to content
Benchmaker

Glossary · Meaning and specification

Construct-irrelevant variance

1Claim

A validity threat in which factors outside the construct, such as output format, token budgets, tool latency, or answer length, drive the score, making it impure. The nuisance list guards against it.

Defined in

Handbook · ClaimConstructs, facets, and nuisancesDefine what the score should respond to, and just as carefully what it must ignore. Your nuisance list is your discriminant test plan.
Nuisance variable
A property that should not materially move the score when the construct is held fixed, such as length, formatting, or assertive tone. The nuisance list is the discriminant test plan.
Construct underrepresentation
A validity threat in which important facets of the construct are absent from the tasks or too rare, so the score is too narrow for its claim. The facet list guards against it.

Where it comes up

See it in the full glossary →