ICLR · International Conference on Learning Representations · 2024

Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stages

Guozheng Ma*, Lu Li*, Sen Zhang, Zixuan Liu, Zhen Wang, Yixin Chen, Li Shen, Xueqian Wang, Dacheng Tao

Editorial illustration of this publication’s research theme — An illustration of the research theme. See the paper for methods and experimental results.

The study examines how data augmentation, network modules and training stages affect plasticity in visual RL. It identifies the critic as a key bottleneck and motivates Adaptive Replay Ratio: adjusting data reuse to the critic’s ability to learn. Read the paper summary.

Guozheng Ma and Lu Li contributed equally.

Watch the AI TIME talk in Chinese.

Read the Chinese coverage.

Related research

Keeping visual RL able to learn

Data, the critic and the timing of updates all shape an agent's learning capacity.