Glossary · Optimization and failure dynamics
Overoptimization
Continued optimization that improves the proxy score after true or reference-anchor quality has plateaued or declined. Gao et al. demonstrated it for learned reward models.
Defined in
Handbook · Validity under pressureGoodhart pressure and self-confirming loopsReward hacking needs an exploit; self-confirmation needs only a shared blind spot. Tell them apart and measure the coupling behind both.Related terms
- Goodhart pressure
- Degradation of the proxy-to-construct relation caused by optimization against the proxy.
- Exploitability curve
- Disagreement between an evaluator and an independent reference anchor as a function of optimization exposure b. b* is the largest tested exposure before divergence exceeds a predeclared tolerance.
Where it comes up
- Exposure, modification, and accessHandbook · Regime
- Evaluators inside training loopsHandbook · Regime