Recent content in Home on Nathaniel Thomas
Many have caught onto the truth that AGI-through-pure-LLM-scaling is probably
not going to happen. And many have identified continual learning as the key
difference between LLMs and a generally intelligent agent. If you’ve ever used
Claude Code, you will be acutely aware of how effective context length limits
LLMs’ general utility, and if only we had something that…
It’s that time of year again. 
 You can try out the solutions here . 
 Day 1 
 module Main where 
 
 import Common ( parseFile ) 
 import Control.Applicative (( <|> )) 
 import Data.List ( foldl' , scanl ) 
 import Data.Text qualified as T 
 import Data.Void ( Void ) 
 import Text.Megaparsec 
 import Text.Megaparsec.Char 
 import…
As we start training LLMs as Agents, we must think about how to best pass
information to and from the real-world environment. If it calls an external
function, how should arguments be passed? How should data from the environment
be fed to the model? The simplest (and most general) solution uses structured
data formats , such as JSON. These formats can encode arbitrarily…
I recently bought Bryan Johnson’s
 Super Veggie T-Shirt ,
in order to fully immerse myself in his protocol. 
 










 

 
 
 

 
 
 It was $37—not a terrible price—and I think it looks cool. But once I receieved
it, I noticed that the quality was markedly better than any other t-shirts I
own,…
Excalidraw currently doesn’t support 
 L A T E X , which sucks. The workaround is to
generate an SVG for whatever math you want to render, and paste that in. 
 You can use this script to generate the SVG: 
 import matplotlib.pyplot as plt 
 
 # use svg backend 
 plt . switch_backend ( 'svg' ) 
 
 # enable latex rendering 
 plt . rcParams […
The Sharpe Ratio measures the quality of an equity or hedge fund by showing the
return per unit of risk, calculated as 
 σ μ − r , where 
 μ 
is the expected return, 
 r is the risk-free rate, and 
 σ is the
standard deviation (volatility). A higher ratio indicates better performance for
the risk taken—more return without excessive variability. In the simulator,…
I find entropy to be extremely fascinating. But, matching the formula

 ∑ p i lo g p i 1 to its “intuitive” explanations related to prefix
free codes and information content is not obvious. Here, I want to go over a
couple ways to independently arrive at the idea.
I’m doing AoC in Haskell to learn the language. These are my solutions. 
 Day 1 
 import Data.List 
 import qualified Data.Map as Map 
 
 f xs = 
 let x1s = sort $ map fst xs 
 x2s = sort $ map snd xs 
 diff x y = abs ( x - y ) 
 in sum $ zipWith diff x1s x2s 
 
 counter = Map . fromListWith ( + ) . map (, 1 ) 
 
 sim xs = 
 let c =…
This entire site is static. All the visualizations are running completely in the
browser. 
 I use Hugo to build the site. It’s pretty neat, since its
template language lets me program a lot features statically, without any
JavaScript. Even the 
 L A T E X on this site is statically rendered! 
 The theme is based off of Typo by tomfran ,
but I’ve made a…
Goal 
 Suppose we have a dataset of features, but no labels. If we know (or guess) that
there are 
 K classes in the dataset, we could model the dataset as the weighted
average of 
 K class–conditional Gaussians. This is what Gaussian Mixture Models
do. 
 We assume that the model is parameterized by

 θ = { π k , μ k , σ k 2 } k = 1 K , where 
 π k …
I am a Neovim diehard, but it is impossible to use over SSH. Since I do ML
research, all my code runs on a remote server with high power GPUs. Reluctantly,
I have been using VSCode, for its excellent remote-ssh plugin. But even with its
half-baked Vim mode, it is still the same sluggish Electron app. 
 Zed may the the editor that changes this game. It is extremely fast,…
Training a deep neural network is essentially a compression task. We want to
represent our training data distribution as a function parameterized by a bunch
of matrices. The more complex the distribution, the more parameters we need. The
rationale for approximating the entire distribution is so that we can forward
 any valid point at inference using the same model, with the same…
Bayesian Parameter Estimation (BPE) is fundamentally different compared to
 MLE or MAP . Whereas the latter two solve for an
optimal set of parameters 
 θ ^ for the model, BPE treats

 θ as a random variable with a distribution

 p ( θ ) . 
 Setup 
 We are given a dataset 
 D , which contains 
 n i.i.d. features

 x j . Given a new feature…
This is a collection of V60 recipes that I have used. 
 Emi Fukahori (1 cup) 
 Source video. 
 This recipe is specific to the Hario switch, my current brewer. It gives a
consistent and bright cup. 
 
 Filtered Water: 200g 
 Coffee: 14g 
 Grind: Medium-coarse, 7.5 on Fellow Ode 2 
 Ratio: 14.28 
 Water temp: 95º C 
 
 
 Close the switch (no flow),…
This is a method of evaluating strategies for the multi-armed bandit problem
 1 . The testbed works as follows: 
 
 Generate 
 10 reward means 
 μ i associated with 
 10 actions 
 a i 
 On each iteration allow the agent to take some action 
 a j , and receive a
reward 
 r t ∼ N ( μ j , 1 ) . 
 
 We repeat this for 
 100 randomly…
The goal is essentially the same as MLE . We have an assumed model
for 
 p ( x j ∣ ω j ) parameterized by 
 θ . We want to classify
a feature 
 x into some class 
 ω j based on a labeled dataset

 D . In MLE, we were trying to maximize the likelihood : 

 θ ^ MLE = ar g θ max p ( D ∣ θ ) In MAP, we instead maximize the a posteriori :
Goal 
 We are given a dataset 
 D , which contains feature vectors

 x k and class labels 
 ω k . Denote 
 D i as the set of
features of class 
 ω i . We assume the following: 
 
 That

 p ( x ∣ ω j ) ∼ N ( μ j , Σ j ) .
That is, given a class label, the distribution of features belonging to that
class forms a Gaussian with mean…
Causal self-attention is the mechanism underpinning most of the advances in AI
since 2017. In this article, I will step through the computation and hopefully
gain a better intuition of how it works. 

 SelfAttention ( Q , K , V ) = softmax ( mask ( d Q K T ) ) V At a high level, this function takes one sequence and transforms it into
another. A sequence is a list of token…
We’re going to go through a minimal example that will let you run Rust code on
the client side of a Hugo site. We are going to compile the Rust code into
WebAssembly (wasm), which will give us near-native performance on the browser!
Explore different methods to win, and beat expert humans in 2048 interactively!
Draw digits on the canvas and watch an AI guess what it is!
One of the most striking elements of Silicon Valley to outsiders is
 productivity culture . Whereas most people in most places live in complete
satisfaction doing their job as they would, Silicon Valley people won’t find
peace without optimizing their every habit and system to extract that extra iota
of productivity per unit time. I am one of those people, and this article…
If you’re a nerd, and you’ve been around Macs for a while, you might remember
Applescript. It was a language developed by Apple to allow
intermediate–to–advanced users to write simple scripts that could control Mac
applications. It was actually created to resemble the English language, so
accessing a pixel would be written as 
 pixel 7 of row 3 of TIFF image 'my…
My previous post (which was honestly created to test out the
theme for this site), provided a few code snippets that computed 
 N terms of
the sum of inverse squares. I wrote the code in my 4 favorite languages—Python,
C, Rust, and Haskell—but when I ran the Python code, it was embarrassingly slow.
Compared to the 
 ≈ 950 ms it took sequential Rust, Python took…
Hello, World! This is my first post, and it’s exclusively used to test out this
website’s functionality. 
 Here are some code snippets in various languages that compute the
 Basel Problem :
I’m a Master’s student at UCSD working on reinforcement learning for Large
Language Models, advised by Prof. Xiaolong Wang . 
 I got started with programming through open source 
in high school. Since then I’ve interned at Anduril, Stanford AI Lab, Keysight,
SDSC, and Yahoo. 
 When I’m not programming, I’m brewing specialty coffee, lifting weights,…
My digital bookshelf, in no particular order.