AI Research

Average-reward RL subtracts a running baseline ρ from every reward. When rewards saturate (converging to a near-constant value), the differential collapses to zero and the agent stops learning entirely. This post dissects the failure mode on a two-pole balancing task, demonstrates a 20× performance reversal by switching from a flat survival bonus to a dense balance-quality signal, and derives a practical reward-engineering checklist for deploying ARPPO in production.

Most RL policies see only the current observation. When sensors drop, signals are noisy, or context spans many steps, a feedforward network is generally insufficient. This post benchmarks LSTM-augmented PPO and ARPPO against feedforward baselines across six partial-observability tasks — and finds a consistent asymmetry between the two algorithms.

Proximal Policy Optimization (PPO) has become the default choice for deep reinforcement learning: reliable, well-understood, and competitive across many benchmarks. But PPO uses discounted reward, which creates a fundamental mismatch with continuing operational tasks. Our ARPPO algorithm [Schneckenreither, 2020; Schneckenreither & Moser, 2025] addresses this directly. Algorithm Comparison Property PPO ARPPO Objective Discounted cumulative reward

Inventory optimization is one of the oldest problems in operations research. Classical solutions such as EOQ, the newsvendor model, and (s,S) policies assume simplified demand distributions and static cost structures. In the real world, demand is seasonal, trends shift, and supplier lead times vary. Reinforcement learning handles all of this naturally . The RL Formulation

When building AI systems for business operations, one of the most consequential design choices is often overlooked: how do you measure success? Most reinforcement learning textbooks and libraries default to discounted reward . For games and simulated environments, this works fine. For real operational problems, it creates a systematic bias that undermines performance. The Discounting