Abstract:What do thinking language models learn during training that their base models lack? We first present an unsupervised method that discovers a model's reasoning behaviors by training small Sparse Autoencoders on sentence-level activations of reasoning traces, yielding interpretable reasoning taxonomies. Building on this, we introduce constructive model diffing, which aims to reconstruct the base-to-fine-tuned difference from interpretable components: reasoning mechanisms (category vectors that can induce a reasoning behavior in the base model) and reasoning heuristics (a classifier determining when a mechanism should fire). Across nine base/thinking pairs (four RL-trained, four SFT-distilled, one mixed), two independent findings agree: category vectors in the base model converge to far lower loss for taxonomies derived from purely RL-trained models, and hybrid models recover roughly 76% of the RL base-to-thinking gap but only 11% of the SFT gap. This indicates RL primarily teaches heuristics for orchestrating pre-existing base mechanisms, whereas SFT-distillation installs new ones, offering a new lens on what training paradigms teach, with implications for efficient reasoning-model development.
| Comments: | Accepted as a Spotlight at the International Conference on Machine Learning 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2510.07364 [cs.AI] |
| (or arXiv:2510.07364v4 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2510.07364 arXiv-issued DOI via DataCite |
Submission history
From: Constantin Venhoff [view email]
[v1]
Wed, 8 Oct 2025 17:58:28 UTC (543 KB)
[v2]
Fri, 10 Oct 2025 17:48:16 UTC (544 KB)
[v3]
Wed, 22 Oct 2025 16:02:22 UTC (544 KB)
[v4]
Tue, 7 Jul 2026 14:19:58 UTC (3,623 KB)