Larger networks do not automatically make better RL agents. Optimization can become harder as capacity grows, so increasing parameter count alone can leave useful capacity out of reach.
The ICML 2025 study tests a simple intervention: randomly remove a fixed fraction of connections once, before learning begins. The remaining network keeps that connectivity throughout training. This static sparsity improves parameter efficiency and helps with plasticity loss and interference between gradients. The experiments also examine visual and streaming RL. Paper and proceedings.
The useful distinction is between available capacity and capacity that can be trained effectively. Sparsity changes that relationship, making network connectivity a meaningful part of the scaling question. The official implementation provides the experimental starting point.