Average-reward RL subtracts a running baseline ρ from every reward. When rewards saturate (converging to a near-constant value), the differential collapses to zero and the agent stops learning entirely. This post dissects the failure mode on a two-pole balancing task, demonstrates a 20× performance reversal by switching from a flat survival bonus to a dense balance-quality signal, and derives a practical reward-engineering checklist for deploying ARPPO in production.