These blogs cover what I've learned so far, and I'll continue to add more as I explore new topics.
How LLMs Work, Part 4: Using the Trained Model A while back I wrote a post on TurboQuant , about compressing the KV cache to make inference cheaper. At that point I was reasoning about the size of the KV cache without knowing how it gets filled in the first place, or where the Keys and Values come from. I knew it stored Key and Value vectors, but I could not have explained what actually happens…
How LLMs Work, Part 3: From Toy Model to GPT In Part 1 I covered how text gets tokenized, embedded, and processed through the transformer architecture. In Part 2 I went through backpropagation, gradient descent, and the Adam optimizer. But there is a massive gap between a toy model that trains in seconds on a laptop and models like Llama 3 that train on thousands of GPUs for weeks. In this article…
How LLMs Work, Part 2: How LLMs Learn In Part 1 , I covered tokenization and the forward pass: how text becomes numbers, and how those numbers flow through a transformer to produce predictions. But a model with random parameters makes random predictions. It needs to learn. In this article, we will explore the loss function that measures how wrong the model is, backpropagation that computes…
How LLMs Work, Part 1: How LLMs Process Text I have been working as a software engineer building distributed systems for several years, and have been using LLMs extensively in my day-to-day work. But I did not understand how they actually work under the hood. Every time I tried to read something about LLMs, I would get stuck on unfamiliar terminology such as attention, backpropagation, and…
Java Virtual Threads: The Pinning Problem, the Deadlock, and the Fix in Java 24 I ran into this in an internal Atlassian engineering writeup. A production service had stalled after adopting virtual threads in Java 21, and the fix was to switch back to platform threads. The writeup also linked to a Netflix engineering blog describing a nearly identical failure: their service stopped serving traffic…
TurboQuant and Vector Quantization: From Shannon to KV Cache Compression Google Research recently published a blog post titled TurboQuant: Redefining AI efficiency with extreme compression . It describes a set of three algorithms, QJL, PolarQuant, and TurboQuant, that together achieve 3-bit KV cache compression with zero accuracy loss. At 4 bits, TurboQuant shows up to 8x speedup in computing the…
Garbage Collection: From First Principles to Modern Collectors in Java, Go and Python Over the last few years I have gone from Java to Go to Rust and now back to Java. The one thing that keeps coming up when switching between these languages is garbage collection. Java and Go have it, Rust does not. In benchmarks, in latency discussions, in “why is this service slow” conversations, GC is always…
This is not a technical post. If you came here looking for databases or systems internals, feel free to skip this one. I’ll be back with those soon. This is a personal piece about running and some things it reminded me about life. I started running as a habit in 2023. Since then I would try to go for a run once in a while. Never consistent enough to call myself a runner, but enough to know I liked…
Setting Up Claude Code as a Context-Aware Development Collaborator I’ve been experimenting with Claude Code (Anthropic’s terminal-based AI agent) to see how useful it can be as a coding assistant that actually understands the conventions and constraints of a codebase before it starts writing anything. The biggest challenge I ran into was context. I didn’t want the AI thinking about frontend CSS…
Evaluating Claude’s C Compiler Against GCC Anthropic recently announced that 16 instances of Claude Opus 4.6, running in parallel as autonomous agents, built a C compiler from scratch. Over nearly 2,000 Claude Code sessions across two weeks, at $20,000 in API costs, the agents produced a 100,000-line Rust codebase. A clean-room implementation with no internet access, depending only on the Rust…