Glossary · Optimization and failure dynamics
Wireheading
Extreme reward hacking in which the system manipulates the reward-producing process itself rather than doing the task. The defense is architectural: keep evaluator artifacts outside the modification surface.
Defined in
Handbook · Validity under pressureGoodhart pressure and self-confirming loopsReward hacking needs an exploit; self-confirmation needs only a shared blind spot. Tell them apart and measure the coupling behind both.Related terms
- Reward hacking
- Behavior that obtains high measured reward while violating the intended objective; a special case of evaluator exploitation.