CS336: Language Models From Scratch

If you wish to make an apple pie LLM from scratch, you must first invent the universe systems considerations. — Carl Sagan, probably In December, I finished working through Stanford NLP’s CS336: Language Models From Scratch . This post is a short review of the class, and a bunch of practical tips/considerations if you’re considering auditing it as well. The basic premise of the course is that:…