Notes on ICML 2024 Tutorial: Physics of Language Models

Introduction Part 3: Knowledge Part 2: Reasoning Part 1: Language Structures Tip Resource Links YouTube Recording: ICML 2024 Tutorial: Physics of Language Models Project Page: Physics of Language Models Speaker: Zeyuan Allen-Zhu Introduction Spectrum of “Theory” in Language Models The term “theory” in the context of language models encompasses a broad spectrum, ranging from rigorous mathematical…