Posted on February 19, 2026 I made two little sites over the past while. Claude Code makes this easy and pretty fun. The first one is an explorer for the token embeddings of LLMs . You can use it to see what tokens are close to what in the embedding space: The qwen3-30b-a3b's output embedding for " Elon" is near various companies Elon Musk owns, his last name, other tech founders, and for whatever…
Posted on March 9, 2024 Despite Betteridge's law , I contend the answer is "probably". I'd prefer to be writing a post without a question mark at the end of the title, but I haven't actually gotten it to work yet. This post is something of a research log documenting my efforts, though written ex post. This code for the models and data processing is here . Why should it be possible? A text-to-image…
Posted on February 21, 2024 As part of a machine learning project to build a text-to-image model , I gathered a large dataset from archives of Imgur. Well, 33 million images in around 11 terabytes sounds big to me, but there are people out there that probably ingest that much in a day. 🤷♂️ For my purposes, it's enough for proof of concept. Anyway, there's some interesting stuff about the…
Posted on September 6, 2021 I've been looking into deep learning libraries recently and JAX seemed interesting, but as far as I could tell no one had actually benchmarked it against PyTorch, the de facto standard. So I decided to implement the same model in both and compare. Here's the top level summary: PyTorch gets 1.11 iterations per second and JAX gets 1.24it/s (12% better) on a Google Colab…
Posted on January 28, 2018 I made 35 predictions last January, and judgement day has come. Well, technically is was the 1st, but I'm writing this today. I'll use evidence as of the 1st where possible. I'm inverting some of them so all the predictions are ≥ 50%. Politics Trump takes office on inauguration day: 95%. TRUE . This one was pretty obvious. Maybe low. Trump is not impeached: 85%. TRUE .…
Posted on December 31, 2016 Here are my predictions for 2017, such as they are. The idea is to develop the skill of prediction by making and testing them somewhat regularly. I don't expect to do super well. Judgment date is January 1 2018. I'm aiming mostly for calibration, but will also compute cross entropy. Politics Trump takes office on inauguration day: 95%. Trump is impeached: 15%. Trump is…
Posted on December 27, 2016 Starting to use Beeminder late in 2015 is, no exaggeration, one of the best decisions I've ever made. I'm much happier and have gotten much more done than I ever did before. I'm gonna be cute and call it Willpower as a Service . If you ever procrastinate or don't do things you know you should, I highly recommend it. Here's how it works: you set a goal to do something…
Posted on August 22, 2016 I uploaded AlanDeniseEricLauren to Hackage today. Here's the README: AlanDeniseEricLauren is an implementation of the ADEL algorithm for efficiently finding the minimal subset of an input set satisfying some arbitrary upward-closed property. "Upward-closed" means if the property is true of some set S it is true of all supersets of S. My implementation is trivially…
Posted on May 19, 2016 I've decided to do an empirical study of the effects of programming language choice. As a PL enthusiast, I'd like to know what languages are actually helpful in practice and whether the ones I like are actually any good. The existing research is, well, not great . In this post, I'll describe my current thoughts. If you have feedback on the design, please leave a comment - I…
Posted on May 10, 2016 I recently spent more than 90 hours debugging what ended up being a problem in GHC's base libraries. I'll explain the bug and the context in which I encountered it, then relate some lessons that would've let me solve it in less than ninety hours. The bug itself In December 2015, I decided to set up a CI build for Idris . I didn't know what I was getting into. Using Stack…