When Speed Kills Stability: Demystifying RL Collapse from the Training-Inference Mismatch

Co-First Authors: Jiacai Liu and Yingru Li Corresponding Authors: Yingru Li and Yu Shen TL;DR The relentless push for faster inference has created a dangerous &ldquo;training-inference mismatch&rdquo; that can silently kill reinforcement learning with LLMs. Our investigation reveals a vicious cycle that is particularly acute in modern reasoning and agentic RL: OOD Contexts Drive Low-Probability…