A Practical (and Incomplete) History of Language Models

This post is a little history lesson in the world of natural language processing. It should give a pretty practical overview of techniques (Markov Chains, BPE, GPTs) used to generate text since 1948 (Claude Shannon’s seminal paper) to today’s LLMs.