TREvaL uses reward models to assess the robustness of generated responses to word-level perturbations. The study examines robustness across language models and observes that supervised finetuning and RLHF can reduce it. The journal article follows the 2023 preprint and SoLaR workshop version.