Writing an LLM from scratch, part 34b -- from bigrams to GPT-2, one component at a time (in JAX)

This post is the capstone of the most long-running series on my blog . In December 2024 (!), I started reading Sebastian Raschka 's book " Build a Large Language Model (from Scratch) ", and worked through it carefully. Being who I am, despite trying to apply a strict "no side quests" policy, I found myself zooming off and digging into all kinds of things. It's time to wrap it up. I had decided…