ICML · International Conference on Machine Learning · 2026 · Spotlight · Top 2.2%

What Makes Value Learning Efficient in Residual Reinforcement Learning?

Guozheng Ma, Lu Li, Haoyu Wang, Zixuan Liu, Pierre-Luc Bacon, Dacheng Tao

Editorial illustration of this publication’s research theme — An illustration of the research theme. See the paper for methods and experimental results.

Residual RL adapts a frozen base policy through bounded action corrections. This work identifies two obstacles for its critic: an uninformed starting point and difficulty distinguishing small residual actions. DAWN addresses them with base-policy data for warmup and critic normalization. Read the conference record.