Personal Blog
Prism: An Impure Functional Language With Typed Effects This is going to be a very nerdy post, so bear with me. Here is the shortest version of the pitch: fn fib ( n ) = var a : = 0 var b : = 1 var i : = 0 while i < n do let t = a + b a : = b b : = t i += 1 a This is a mutable loop with the type Int -> Int . The mutation is real, but it is local and unobservable, so the function is pure from the…
Crypto in 2026: Oh, This is the Bad Place Open your favourite Doom News App any morning in 2026 over your morning coffee, and the only honest phrase that should come out of your mouth is, "Oh, this is the Bad Place." Because the world has gone truly mad. The president of the United States is running a memecoin out of the White House, and the top two hundred and twenty holders were flown to his…
Book Review: On the Calculation of Volume Solvej Balle's On the Calculation of Volume is a planned septology about a Danish antiquarian book dealer who falls out of time, and the first five volumes are one of the most original and brilliant literary projects I've read in years. The premise is the one you have seen a hundred times. The protagonist, Tara Selter, wakes up on the eighteenth of…
A Field Guide to Bugs Software bugs predate software. Edison used the word in an 1878 letter, eighty years before the Harvard moth and sixty before the modern computer. What he named has outlasted him. Every engineer eventually assembles a private taxonomy of the ways things fail, and the useful fact about these private taxonomies is that they converge. Engineers who have never met, working on…
Book Review: What We Can Know Ian McEwan has described What We Can Know as "science fiction without the science," which is both a fair warning and a precise advertisement. The novel is set in 2119. Rising seas and a cascade of nuclear wars (including a misfired Russian warhead in the Atlantic and something called the Third Sino-American War) have halved the world's population and turned Britain…
Book Review: There Is No Antimemetics Division There is a particular flavor of horror that only people who work with formal systems for a living can fully appreciate. It is the horror of data loss, of silent corruption, of the thing that fails without logging an error. It is the backup that was never tested. The monitoring system that monitors everything except its own health. The silent failure…
Book Review: Piranesi Susanna Clarke's Piranesi is a book about a man who lives in a House. The House has an infinite number of marble halls. Ocean tides flood the lower floors. Clouds drift through the upper ones. Thousands of statues stand in the middle halls, each one unique, depicting figures from mythology and daily life and everything in between. The narrator maps the halls, tracks the…
Optimal Caverna Gameplay via Formal Methods I always win at Caverna (Uwe Rosenberg's classic European worker placement tabletop board game). Always. But "always" just means "every time so far," and I needed something with more mathematical permanence. So I formalized the entire game in Lean 4 and proved that my strategy is the unique weakly dominant pure strategy across every possible game…
Can Opus 4.6 do Category Theory in Lean? I have a little category theory library I've been dragging around for about a decade now. It started life in Haskell, got ported to Agda, briefly lived in Idris, spent some time in Coq, and has now landed in Lean 4. I call it Kitty Cats . The idea is simple: take the definitions and theorems from Awodey's Category Theory and the relevant chapters of Mac…
Book Review: When We Cease to Understand the World Benjamin Labatut's When We Cease to Understand the World is the best book I have read in years, and it is the kind of book that makes you feel slightly unwell afterward in the way that only the best books can. It is nominally about the history of science. It is actually about what happens to the human mind when it touches something it was never…
Hypothetical Divine Signatures The author of the Epistle to the Hebrews famously claimed that "faith is the substance of things hoped for, the evidence of things not seen," which was a perfectly serviceable theological patch for an era where the average person’s greatest computational challenge was counting their own fingers. However, in an age where we can simulate galaxies and sequence genomes,…
Typechecker Zoo As a fun side project, I've built four literate Rust implementations spanning 50 years of type theory research, from Milner's Algorithm W to modern dependent type systems. I've tried to make each implementation as readable and understandable as possible, with detailed explanations and comments throughout the code and to make it as succinct and short as possible so you can pull it…
The Stochastic Code Monkey Theorem The relentless hype machine surrounding large language models would have you believe we are on the cusp of a software development revolution, a new epoch where programmers are obsolete, their keyboards gathering dust while throwing enough GPUs and parallel stochastic code monkeys at any problem will conjure the coding prowess of Fabrice Bellard and Jeff Dean…
Christian Nationalism Is A Dangerous Ideology Every few decades, one of America's two political parties undergoes a structural realignment so complete that the continuity of the name becomes a polite fiction. The Democrats did it in the 1960s. The Republicans are doing it now. Reagan's "three-legged stool" of social conservatives, economic conservatives, and defense hawks, the coalition that…
It Would Be Good if the AI Bubble Burst As a software engineer, it is impossible to ignore the strangeness of the current moment. On one hand, the new generation of language models are genuinely useful tools. I use them myself to get a head start on boilerplate code, to brainstorm solutions to tricky logic problems, or to rephrase documentation. They are a legitimate, if incremental, step forward…
Every Democrat Who Enabled Trump's Crypto Corruption The Democratic Party will tell you, at considerable length and with great sincerity, that it stands for consumer protection, financial regulation, and democratic accountability. What it actually does is a separate matter. Then a sitting president launched his own memecoin, installed industry loyalists to gut the agencies meant to police fraud,…
Only Solution to the Attention Economy is to Opt Out There is a spectacular, almost beautiful absurdity in modern life, blithely plugging our ancient, analog brains into a digital world that wants to eat them for lunch. Our grey matter, honed over eons to spot a lion in the savanna or find a decent berry patch, is now the unwilling participant in a cage match against algorithms designed to turn…
Fast Tensor Canonicalization in Rust I recently had a need to do fast tensor canonicalization, but the library ecosystem for this was lacking. In physics and differential geometry, manipulating tensors and especially bringing them into canonical form under their index symmetries is a recurring and computationally intensive task. Every undergrad who takes their first relativity class understands…
Fine-tuning With Tool Calling With the release of trl 0.19.0, there is now native support for fine-tuning models with custom tools, a capability that has been seamlessly integrated into the Dataset and SFTTrainer classes. This streamlines the training process of teaching a model how and when to call external functions. We can now structure our training data to include the full conversational flow…
Program Synthesis: The λ in the Machine Much of the "mainstream" discussion of artificial intelligence applied to code generation is overwhelmingly dominated by the idea of scaling larger langauge models and using test-time compute to improve model performance on coding ( SWE-bench , LiveCodeBench, etc) benchmarks. And that has legitimately produced some very interesting results. Although I'm not…
The Asbestos Doctrine: Why Crypto Must Be Contained, Not Condoned For more than a decade, the public discourse surrounding cryptocurrency has been trapped in a fog of misleading analogies and deterministic prophecies. Its defenders have presented it as a misunderstood marvel, variously portrayed as the next iteration of the internet, a lifeboat for the oppressed, or a necessary evolution in…
What is AI actually worth? The artificial intelligence boom is obviously the latest historical example of a market bubble, but it is a particularly strange one. It is a bubble inflated by a potent mixture of managerial fantasy, quasi-religious fervor, and a torrent of capital with nowhere else to go. While proponents and industry thought leaders herald a new economic dawn that rivals the…
Book Review: Tomorrow, and Tomorrow, and Tomorrow Gabrielle Zevin's Tomorrow, and Tomorrow, and Tomorrow is a novel about two people who make video games together for thirty years, and if that sentence made you instinctively reach for the back button, I understand, but you would be making a mistake. The book is not about the games industry in the way that industry memoirs are about the games…
The Future of Maths May Be Deeply Weird In fiction there is a popular image of a mathematician as a lone solitary figure, a figure of lost in their own world in a world of intense concentration before a chalkboard dense with arcane symbols. That may have had some truth in the past but the reality is that mathematics these days is a far more social phenomenon and future mathematicians may have a…
Does the Star Trek Computer Run on COBOL? We like to imagine the future as sleek, seamless, and intelligently designed — a world where a single voice query to a starship's LCARS interface dispatches 575 trillion calculations per nanosecond through the bio-neural gelpacks (biological FPGAs), to run a massively parallel computation to calculate the optimal molecular dynamics for the replicator to…
Interfacing MCP with Combinatorial, Convex, and SMT Solvers Lately I've been working on using MCP beyond just using it for symbolic algebra manipulations , I've been thinking about how to interface large language models with a suite of dedicated solvers for scientific computing exposed as tools to the model. And particularly for automating workflows in physical engineering disciplines. Github…
The Trump Crypto Orgy of Corruption We are living in very unusual times and corrupt times. Never before has an American president been so brazen in his corruption and self-enrichment. This new phenomenon, what Senator Elizabeth Warren coined as an "orgy of corruption," is staggering in scope, and while I'm under no illusion that writing about this will be news to anyone, I feel compelled to…
Unnecessary and Unstable: Why Stablecoins are Bad The current financial policy debate in the US is increasingly fixated on the purported necessity of bespoke regulation for stablecoins (particularly the proposed S.394 GENIUS Act ), with proponents arguing that such frameworks would foster responsible innovation and integrate these digital tokens into the broader financial system. This perspective,…
Remote MCP Servers After my last adventure with symbolic algebra and MCP , I found myself that nagging voice in the back of my head whispering "What if Claude decides today is the day to go postal and rm -rf / ?" The solution, as with most modern software problems, was obvious: make it someone else's problem. Enter Vercel , purveyor of the finest VC-subsidized free compute known to humanity. But…
Adventures in Symbolic Algebra with Model Context Protocol I spent last weekend playing with this new MCP protocol all the kids are talking about, using it to make language models talk to symbolic computer algebra systems. The idea was simple: LLMs are great at understanding natural language math problems but terrible at actually solving them, while computer algebra systems excel at symbolic…
Using CUDA Deep Neural Network (cuDNN) in Python Let's go through how to implement scaled dot product attention using the cuDNN Python API. This is the most computationally expensive part of inference in a transformer-style model, while also being partially parallelizable so it's usually offloaded to the GPU. Under the hood this uses the FlashAttention-2 algorithm but provides an API that is…
The Technofascist Mind: A Guide to Its Psychology and Philosophy I often read the usual American coastal elite magazines—the New Yorker , Guardian , the Atlantic , and others—and experience a profound sense of vertigo when opinion writers diagnose the emergent ideologies of Silicon Valley's power brokers as 'anti-democratic' or 'illiberal' as if this is some new revelation. Yes, obviously. In…
The Kardashev-Marx Scale Okay, buckle up, fellow cogs in the great Techno-Capital machine! Forget the starry-eyed optimism of the original Kardashev scale—that relic of a bygone era where 'progress' was defined by pure consumption and didn't imply more advanced ways to feel empty inside, we now have a Modest Proposal for a new scale. We present the Kardashev-Marx Scale which charts civilizational…
AI Slopocalypse 2027 We predict that the impact of superhuman AI slop over the next decade will be enormous, exceeding both the invention of self-replicating molecules and Cheese Wizz. We (i.e the Royal We) wrote a scenario that represents our best guess about what that might look like. It's informed by science fiction, trend exaggerations, video games, echo-chamber feedback, vibes, and the…
Attention Wasn't All We Needed There's a lot of modern techniques that have been developed since the original Attention Is All You Need paper. Let's look at some of the most important ones that have been developed over the years and try to implement the basic ideas as succinctly as possible. We'll use the Pytorch framework for most of the examples. Note that most of these examples are highly…
How Democrats Failed on Crypto It's hard to look back at the previous US administration and not recognize the near complete regulatory failure on the crypto issue. This failure has now metastasized into a serious political machine on Capitol Hill, becoming a giant fountain of corruption that pools dark money from unknown sources abroad into our politics and fuels far right extremist movements.…
GPU Compilation with MLIR Continuing on from last time, now that we have our transformer primitives that we need to optimize, let's look at how we can translate high-level tensor operations (like softmax , attention , etc) expressed in MLIR down to low-level code that can run in parallel on Nvidia GPUs to improve our performance. To do that we're going to start to put together the making of a…
Transformers The transformer architecture revolutionized the field of natural language processing when introduced in the landmark 2017 paper Attention is All You Need . Breaking away from traditional sequence models, transformers employ self-attention mechanisms (more on this later) as their core building block, enabling them to capture long-range dependencies in data with remarkable efficiency.…
Specializing Python with E-graphs We've explored progressively more sophisticated techniques for optimizing numerical computations. We started with basic MLIR concepts, moved through memory management and linear algebra, and then neural network implementations. Each layer has added new capabilities for expressing and optimizing computations. Now we're reading to build our first toy compiler for…
Neural Networks In our journey through MLIR and modern compiler technology, we've explored how to build efficient systems for numerical computation. Now, we turn our attention to one of the most important applications of these systems: deep learning. Before diving into how MLIR can optimize neural network computations, it's crucial to understand how neural networks work from first principles. This…
Linear Algebra in MLIR The Linalg dialect is a core idea in the MLIR ecosystem, it was built to make high-level tensor computations easier and more efficient. Essentially, it's all about simplifying the way we handle complex mathematical operations in a way that can be translated into fast, optimized code. At its core, the Linalg dialect uses a straightforward, declarative approach. This means it…
Affine Dialect and OpenMP The affine dialect in MLIR provides abstractions for expressing loops and array accesses that are amenable to parallel execution. Unlike the scf dialect we've seen before, the affine dialect enables advanced loop transformations and automatic parallelization. It's esesntially "first class loops" which is something we always used to layer on top of our LLVM. The affine…
Memory in MLIR MLIR's memory architecture represents a balance between high-level abstractions and low-level efficiency. At its core, MLIR provides three primary ways to represent data: Tensors for immutable, abstract operations; MemRefs for concrete memory buffers; and LLVM-level constructs for fine-grained control. This three-layer approach allows developers to work at their preferred level of…
Introduction to MLIR LLVM is a powerful compiler infrastructure project that has revolutionized how we build programming languages and tools. At its core, LLVM provides a collection of modular compiler and toolchain technologies that can be used to develop frontend compilers for any programming language and generate optimized code for many target architectures. MLIR is a newer project within the…
MLIR Part 0 - Installing MLIR Installing MLIR can be a royal pain, so here are some notes on how to do it via various methods. From Homebrew (macOS) Using Apt (Ubuntu 22.04) Using llvm.sh Precompiled Packages From Source (macOS) From Source (Ubuntu 22.04) From Wheel (pip) From Wheel (Poetry) From Wheel (uv) Using Conda Using Docker Installing MLIR Python Bindings From Homebrew (macOS) The simplest…
SGLang on AMD MI300X The best kept secret these days is that new AMD GPUs are incredibly cost efficient for doing inference. If you want the cliff-notes on getting an extremely fast inference server running on a fleet of AMD MI300X, here you go: If you're serving high-throughput inference, your options for inference servers are essentially: TensorRT-LLM : NVIDIA's high-performance deep learning…
How to Make a Universe There's a new paper How to Make a Universe by Bassani and Magueijo on the quantum cosmology arXiv that proposes a curious and clever new cosmology. They propose that the fundamental constants (\(c, G, \hbar, k_B, e\)) are not fixed numbers, but instead evolve through a process akin to natural selection. Unlike the cosmological natural selection proposed by Smolin , this…
The FlashAttention CUDA Kernel Line by Line Flash Attention is a memory-efficient algorithm for computing attention in transformers. Let's break down the CUDA implementation block by block. The core innovation of Flash Attention is processing attention in blocks while maintaining numerical stability through careful tracking of maximum values and partial sums. The algorithm achieves memory…
Training with GRPOTrainer DeepSeek's impressive new reasoning model is powered by a key innovation called Group Relative Policy Optimization (GRPO). This clever variant of PPO not only improves the model's training convergence but also uses less memory than traditional approaches. This was one of the core idea that helped DeepSeek's new R1 model achieve state-of-the-art performance with incredibly…
The Case Against Crypto in 2025 In 2020, I wrote what became a widely-circulated critique of cryptocurrency and its implications for our financial system. At the time, I desperately hoped to be proven wrong. As a technologist who deeply believes in the potential of technology to strengthen democratic institutions, I took no pleasure in forecasting the corrosive potential of crypto assets.…