How I trained my own GPT-2 and fine-tuned it into an instruct model

This is the second blog post about LLM. All along, I had been fine-tuning models but never really trained a model from scratch. I then got recommended the video from Andrej Karpathy about LLM Deep Dive, where he introduced a few major steps to build a large language model. So this blog post is about how I trained my GPT-2-like model from scratch and fine-tuned it into an instruct-following model.…