The Map
The series as a dependency graph The whole premise of the series is that the LLM stack is a dependency graph, and that nothing should get used before the intuition for it has been earned. Every relationship here comes from the posts themselves. Select a concept to mark the earlier material it uses and the later posts that use it, then walk backwards until you reach ground you already know.
Select a post to trace it
1.1 Vectors, Matrices, and the Spaces They Live In 1.2 Norms, Dot Products, and Similarity 1.3 Distributions, Softmax, and the Chain Rule of Words 1.4 Cross-Entropy, KL Divergence, and What Loss Functions Measure 1.5 Gradients and How Machines Learn 1.6 Optimizers: Momentum, Adam, and Learning Rate Schedules 1.7 GPUs, Floating Point, and Why Precision Matters 1.8 A Short Prehistory of Statistical NLP ARC 02 Before Transformers 2.1 Language Modeling as Next-Token Prediction 2.2 N-gram Models and the Curse of Sparsity 2.3 Word2vec and the Embedding Revolution 2.4 GloVe, FastText, and the Embedding Zoo 2.5 Recurrent Neural Networks and Sequence Modeling 2.6 LSTMs, GRUs, and Gated Memory 2.7 Seq2seq, Bahdanau Attention, and Why Recurrence Hit a Wall 3.1 Unicode, Bytes, and What Text Actually Is 3.2 Byte-Pair Encoding from Scratch 3.3 WordPiece, Unigram, and SentencePiece 3.4 Vocabulary Size, Merge Order, and Fertility 3.5 The Embedding Table and Its Geometry 3.6 Special Tokens, Chat Templates, and Input Formatting 3.7 Packing, Masking, and Tokenization as a Model Interface 4.1 Self-Attention: Q, K, V from First Principles 4.2 Multi-Head Attention and Representation Subspaces 4.3 Causal Masking and the Autoregressive Constraint 4.4 Positional Encodings: Sinusoidal, Learned, RoPE, and ALiBi 4.5 The Feed-Forward Block as Key-Value Memory 4.6 Layer Normalization: Pre-Norm, Post-Norm, and RMSNorm 4.7 Decoder-Only vs. Encoder-Decoder: Architecture Trade-offs 4.8 A Close Reading of ‘Attention Is All You Need’ 4.9 Mixture-of-Experts Layers ARC 05 Decoding & Inference 5.1 Training View vs. Inference View 5.2 Prefill vs. Decode: The Two Phases of Inference 5.3 Why One New Token Means One New Row 5.4 The KV Cache from First Principles 5.5 Sampling Strategies: Temperature, Top-k, Top-p, and Min-p 5.6 Speculative Decoding 5.7 Continuous Batching 5.8 Prefix Caching and Prompt Reuse 5.9 Structured and Constrained Generation 6.1 What an Inference Engine Actually Does 6.2 Kernel Fusion and the Memory Wall 6.3 PagedAttention: Virtual Memory for the KV Cache 6.4 Memory Management: Fitting a Model and Deciding Concurrency 6.5 Quantization: INT8, INT4, GPTQ, AWQ, and GGUF 6.6 Tensor, Pipeline, and Expert Parallelism 6.7 Throughput vs. Latency: Picking Your Tradeoff 6.8 Comparing Engines: vLLM, TGI, TensorRT-LLM, llama.cpp, SGLang 6.9 Benchmarking Your Own Inference Stack 7.1 Pretraining Data: Mixtures, Curation, and What the Model Sees 7.2 Distributed Training: FSDP, DeepSpeed, and Megatron 7.3 Supervised Fine-Tuning 7.4 The RLHF Pipeline: Reward Models and PPO 7.5 DPO and Its Variants: Skipping the Reward Model 7.6 Constitutional AI and RLAIF 7.7 Tool-Use Fine-Tuning 7.8 Long-Context Training Techniques 7.9 Synthetic Data and Distillation 7.10 LoRA and Parameter-Efficient Fine-Tuning 8.1 Loss vs. Benchmarks: Two Languages for Model Quality 8.2 Eval Harnesses: lm-eval-harness and HELM 8.3 Contamination and Leakage 8.4 Evaluating Reasoning 8.5 Tool and Agent Evaluation 8.6 Human Preference Evaluation 8.7 Calibration and Abstention ARC 09 Retrieval & Context 9.1 Embeddings from Scratch: From Word2Vec to E5 9.2 Vector Search and Approximate Nearest Neighbors 9.3 Dense, Sparse, and Hybrid Retrieval 9.4 Chunking Strategies 9.5 Rerankers and Cross-Encoders 9.6 RAG Architectures End to End 9.7 GraphRAG and Knowledge Graphs 9.8 Context Windows: Advertised vs. Effective 9.9 Context Engineering: Compaction, Clearing, and Memory 10.1 A Short History of Agents: From ReAct to 2026 10.2 Function Calling as Structured Generation 10.3 The Agent Loop: Model, Runtime, Tool, Resume 10.4 Transcript Formats: How Providers Represent Tool Use 10.5 MCP: A Cross-System Standard for Tool Integration 10.6 Remote MCP, OAuth, and Enterprise Auth 10.7 Planning and Reasoning in Agents 10.8 Computer Use and Browser Agents 10.9 Multi-Agent Systems and Agent-to-Agent Protocols Select a post to mark its prerequisites in blue and later dependents in amber. Dashed rings mark the featured posts.
Prefer a list? The series index has all ten arcs with curated starting paths.