The Last Mile of LLM Inference
(Part 3) Sampling strategies and why inference optimizations pose a security tradeoff
Ani's Blog
Hey there! My name is Anirudh. I like figuring out how things work — and writing about it now and then.
Latest posts
⭐Why Your First Token Is Always Late
(Part 2) The inference side of transformers, along with systems tricks that make production LLMs fast
You're Billed by the Token. Here's What That Means.
(Part 1) BPE Tokenizers under the hood, and where tokenization breaks math, spelling and code