Recap on implementing GPT model from scratch

TL;DR I am recaping blog post Implementing a GPT model from scratch by answering workbook questions Build llm from scratch chapter 4, and reading Giles blog posts on this chapter: part15 part16 part17 part18 part19 Giles’ blog posts I got lost during Giles’ part 15 on the first read. The second read made things clearer. He goes deep into the GPTModel architecture and how data flows from tokens to…