Notes on Language Models

Towards a Theory for Large Language Models from Across the Internet A concrete theory for large language models must start with the phenomenon that resulted in the notion that such large models could exist . There are small disagreements on which phenomenon that is, but arguably the first observation that hinted towards the existence of such models was what we call the double descent -- the…