M3DT combines a mixture-of-experts Decision Transformer with a staged training procedure to improve multi-task scalability. The approach separates task demands across parameter subsets and studies how increasing the expert capacity supports learning across larger task collections. The evaluation distinguishes scaling the number of tasks from simply expanding a model on fixed tasks.
An earlier version was presented as an oral at the MCDC Workshop at ICLR 2025.