Procedural Knowledge in Pretraining Drives LLM Reasoning

This page cannot be shown here. You can still read it on the original site — the toolbar below keeps your place in the directory.

Authors . Laura Ruis , Maximilian Mozes , Juhan Bae , Siddhartha Rao Kamalakara , Dwarak Talupuru , Acyr Locatelli , Robert Kirk , Tim Rocktäschel , Edward Grefenstette , Max Bartolo TL;DR . LLMs strategy for mathematical reasoning looks unlike retrieval from the parametric knowledge formed during pretraining. Instead, the models learn to apply procedural knowledge extracted from documents…

Read on /2024/11/10/if.html ↗