This study compares reinforcement finetuning algorithms and parameter-update choices for Transformer-based agents using few-shot offline data. It examines multiple meta-RL environments and combines supervised and reinforcement objectives to improve adaptation. The analysis explores when reinforcement objectives add value beyond supervised finetuning, rather than assuming one adaptation strategy is uniformly effective.

Publication record.