Don't Diffuse, Speculate
Diffusion and speculative decoding both trade breadth for depth. In most circumstances, speculative decoding is the better way to make that trade!
Joe Barrow
Latest posts
A Visual Guide to the Roofline Model
My attempt at a simple/visual explanation of arithmetic intensity and the roofline model, and why they matter for making LLMs go fast.
Book Review, Inference Engineering
Inference Engineering by Philip Kiely is an excellent resource and if you think you need to read it, you probably need to read it.
Searching, Fast and Slow
Revisiting 'Slow Search' in the age of agentic retrieval.
Building a 1-Outlet, 4-GPU Workstation
As a grad student, the one thing I desperately wanted was a GPU workstation. As a gainfully employed adult I can finally make that happen.
Agents have (Information) Needs
Information retrieval is about satisfying an information need, but a query is a poor stand-in. Your agent is capable of expressing one, so you should probably use it.
OCR's Cambrian Explosion 1 - Introduction
Strong, open VLMs enabled an explosion of open OCR model releases, with little sign of things letting up. In this survey, I detail the models, their evaluation, research trends, and open questions.
Google Gemini 102 - Advanced Structured Outputs
Probing the supported output types of Gemini.
Google Gemini 101 - Object Detection
Navigating Gemini's API for object detection with vision and Structured Outputs.
Be Careful Interpreting Averaged Benchmarks
Thoughts on averaged benchmarks and hidden correlations.
Introduction to TinyHNSW
The first post in the TinyHNSW series, introducing the tutorial and the library.