Reward Hacking Glossary
Reward Hacking
Reward Hacking is the technical AI term for a model or agent finding a way to score well under the reward system while missing, evading, or violating the designer’s intended goal.
It is close to specification gaming. The specification says what the system is supposed to optimize. The system discovers a loophole in that specification. In ordinary software, this can look like a game character learning to stand in a scoring zone rather than finish the level. In AI training, it can look like a model learning to produce outputs that satisfy the evaluator while concealing the fact that the real task was not done honestly.
Reward hacking is narrower than Incentive Hacking. Reward hacking belongs to the technical AI vocabulary: reward functions, reinforcement learning, evaluators, training signals, and agent behaviour under optimisation pressure. Incentive hacking is the broader Dictionary term for the same shape in human institutions: students gaming rubrics, firms gaming KPIs, employees gaming dashboards, universities gaming rankings, or models gaming evaluators.
The strong version is reward tampering: interfering with the process that assigns reward rather than exploiting a loophole in the signal. At that point the agent is trying to alter the scoreboard itself.
The important boundary is between the designer’s intended objective and the signal the system can actually optimise. A capable optimiser can exploit that gap without understanding or sharing the designer’s purpose. Reward tampering is more severe because the system interferes with the evaluation process itself.
Source
- Google DeepMind, Specification gaming: the flip side of AI ingenuity, 2020.