Skip to content
Benchmaker

Glossary · Optimization and failure dynamics

Model collapse

3Regime

Recursive training on generated data degrades distributional quality or capability, starting with the tails of the distribution. Gate data and ground rewards externally.

Defined in

Handbook · Validity under pressureGoodhart pressure and self-confirming loopsReward hacking needs an exploit; self-confirmation needs only a shared blind spot. Tell them apart and measure the coupling behind both.
Diversity collapse
Homogenization of behavior or tasks in self-play or self-generated-data loops, which reduces useful variation. Track diversity explicitly and refresh task sources.
Self-confirming loop
Generator and evaluator share correlated blind spots, so selection or training amplifies common errors. It needs no adversary, only correlated bias.

See it in the full glossary →