Skip to content
Benchmaker

Glossary · Objects and system context

Evaluator agent

An agent that administers an assessment: it prepares the environment, sends tasks to participant agents, collects results, and reports the metrics it defines. It bundles harness, specification, and mechanism; Berkeley's lecture calls it a green agent.

Also called: green agent

Defined in

Handbook · LifecycleHarness architecture and reproducibilityThe harness is part of the instrument. Isolate trials, pin environments, version judges, and log enough to turn a surprise into a result.
Eval harness
The machinery that administers an evaluation and records trials. It is part of the instrument, distinct from the agent harness, which is part of the system under test.

Where it comes up

See it in the full glossary →