How LLMs Work, Part 4: Using the Trained Model A while back I wrote a post on TurboQuant , about compressing the KV cache to make inference cheaper. At that point I was reasoning about the size of the KV cache without knowing how it gets filled in the first place, or where the Keys and Values come from. I knew it stored Key and Value vectors, but I could not have explained what actually happens…