Personal website for Finbarr Timbers
There's a Google PDF with a bunch of useful latency numbers that software engineers should know. I asked ChatGPT to make it into a markdown file for me. Latency Table Operation Time (ns) Time (ms) L1 cache reference 1 Branch misprediction 3 L2 cache reference 4 Mutex lock/unlock 17 Main memory reference 100 Compress 1 kB with Zippy 2,000 0.002 Read 1 MB sequentially from memory 10,000 0.010 Send 2…
Most ML code is Python. This is surprising to many performance oriented engineers coming from non-ML communities. Python is, notably slow and has the GIL, which forces it to only execute a single thread at a time. The GIL primarily exists to make CPython’s memory management thread-safe. It has also become a load-bearing part of Python's API, as, in a classic example of Hyrum's Law , much Python…
For Olmo 3, I was put in charge of our post-training infrastucture. We made the decision to move from a synchronous RL setup to an asynchronous one to enable us to scale. In doing that work, I was fortunate enough to find a series of optimizations which made our RL setup 4x faster. As we used roughly 250k H100 hours running RL on Olmo 3, these optimizations saved us approximately 750k H100 hours…
The Bitter Lesson is an excellent essay which is overwhelmingly misunderstood. The point of the bitter lesson is that, over time, methods which scale with compute will outperform methods that do not. It is not: The idea that we should never incorporate human knowledge The idea that deep learning and scale are all we need (Rich is actually relatively skeptical of deep learning) The entire point of…
In the recent wave of research studying reasoning models, by which we means models like O1 which are able to use long streams of tokens to "think" and thereby generate better results, MCTS has been discussed a lot as a potentially useful tool. However, some papers, like the DeepSeek R1 paper , have tried MCTS without any success. I worked closely with MCTS for several years while at DeepMind, and…
A few people have asked me for RESP advice, so here is my generic answer, with the disclaimer that I'm not a financial advisor. There are two major decisions to make for an RESP: Decide how much you want to invest per year Decide whether you want to self-manage your money or not This article discusses the options for 1. It really depends on how much money you have available to invest and what your…
I consistently run into the same issue when install Docker on a new Ubuntu VM. I copy the command from the Docker docs , and it fails with the error: E: The repository 'https://download.docker.com/linux/ubuntu bookworm Release' does not have a Release file. N: Updating from such a repository can't be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository…
I recently gave a talk at Amii about the history of GPT models. The slides are now online , as is the recording .
In the DALL-E paper , we want to model the joint likelihood of the model distribution over images \(x\), captions \(y\), and tokens \(z\) for an encoded RGB image 1 . The joint likelihood is modelled using the factorization \begin{equation} \label{eq:factorization} p_{\theta, \psi}(x, y, z) = p_\theta(x | y, z) p_\psi(y, z) \end{equation} The paper presents this lower bound \begin{equation}…
If you want to read more of my writing, I have a Substack . In this article, I discuss the generative pre-trained transformer (GPT) line of work, and how it has evolved over time. I focus on the SOTA models, and the differences between them. There are a bunch of different articles summarizing these papers, but nothing that I’m aware of that explicitly focuses on the differences between them. I…