Rollout Entropy Dynamics

Introduction What is being asked in these experiments is if some future information, in the form of a rollout being successful or unsuccessful, is "leaked" (in the form of entropy over time) somewhere early enough in a completion that by intervening mid-generation or otherwise you can induce likely-to-fail rollouts into becoming successful. Background The experiment was done using TRL's AsyncGRPO…