Doom, compiled into a transformer
No training anywhere: a compiler sets the weights of a stock Phi-3-architecture checkpoint so autoregressive generation executes Doom's classic renderer.
Out of Distribution
Notes from the tail — long-form, interactive writing on transformers and computation by Rob Porter.
Latest posts
A calculator, compiled into a transformer
Setting a transformer's weights by hand -- no training -- to build four exact calculators and compare how they use layers, parameters, and generated tokens.
Introducing torchwright
A compiler that turns computation graphs into the weights of a stock transformer -- no training anywhere. Where it came from and how it works.
What does a "difficult" MNIST digit look like?
The MNIST digits a simple convnet gets wrong, and what they suggest about how the state of the art compares to human performance.