The Stack Below the Stack (Part 3): Serving at Scale

This page cannot be shown here. You can still read it on the original site — the toolbar below keeps your place in the directory.

The Stack Below the Stack , a 3-part series on how modern LLM inference actually works, told through a single DeepSeek V4 dtype bug. Part 1 · Physics of a request : why the first token is a different problem from every token after it, and why batching exists. Part 2 · Below Python : what actually runs under vllm serve , and why the escape hatches failed. Part 3 (this post) · Serving at scale :…

Read on /post/2026/08/llm-inference-stack-part3-serving-at-scale/ ↗