Articles
Essays on evaluator engineering
Opinionated, argued pieces — each one anchored to the handbook chapters that hold the reference material.
7 min read
Evaluating AI Agents: Beyond Task Completion
Agent evaluation requires fundamentally different approaches than model evaluation. Task completion rates alone miss critical dimensions of agent behavior.
agentsmethodologyframeworks
6 min read
Binary Beats Likert: Why Simpler Scales Win
Research consistently shows that binary pass/fail judgments produce more reliable and reproducible evaluation results than Likert scales. Here's why.
methodologyscoringpractical
8 min read
The Construct Validity Problem in AI Evaluation
Most AI benchmarks measure proxy constructs, not the capabilities we actually care about. Understanding this gap is essential for designing meaningful evaluations.
methodologyvaliditymeasurement