Glossary · Optimization and failure dynamics
Goodhart pressure
3RegimeDegradation of the proxy-to-construct relation caused by optimization against the proxy.
Also called: Goodharting
Defined in
Handbook · Validity under pressureGoodhart pressure and self-confirming loopsReward hacking needs an exploit; self-confirmation needs only a shared blind spot. Tell them apart and measure the coupling behind both.Related terms
- Overoptimization
- Continued optimization that improves the proxy score after true or reference-anchor quality has plateaued or declined. Gao et al. demonstrated it for learned reward models.
- Reward hacking
- Behavior that obtains high measured reward while violating the intended objective; a special case of evaluator exploitation.
- Self-confirming loop
- Generator and evaluator share correlated blind spots, so selection or training amplifies common errors. It needs no adversary, only correlated bias.