A cozy little corner of the web.
It’s been a while, I know. I sit here writing today because I feel trapped; a prisonless prisoner who laid bricks until one day he woke up in a cell of his own creation. Am I overconstrained, overburdened, overpromised? In the event that I carry all the parcels I bear through to the date of their respective deliveries, I’m certain I will emerge on the other side a better person. The constraints…
Slowly emerges from the sixth sea of silence. Hello! I’m back for the time being. I started working on a handful of projects, and those have been taking up my blogging time. Sorry for the inconsistency. I wanted to announce that I’ve published the first (and hopefully only) version of Neopack! Neopack is a very small library for quick and schema-agnostic tag-length-value serialization. It should…
These past two days have been something of a whirlwind, as I made a couple of important life decisions. The decisions were, in a strange way, very small, but in a “marble rolling down a saddle” sort-of way that might compound and drastically change the side of the saddle my timeline will spill down… we shall see! I’ve repeatedly brought up the paper Programming as Theory Building in conversation…
After yesterday’s massive post (and given the two assignments I have to do today) I figured I would write something a little shorter today. A little observation, perhaps. In the 70s and 80s , people cared a lot about symbolic reasoning; precise relational systems of facts and rules used to derive logical truths. Any relational system is one concerned with resolving constraints. These ideas fell…
This is a follow-on post to yesterday’s , um, last Friday’s post about curiosity driven approaches to reinforcement learning. I intended to publish this post on the 14th, but it was Valentine’s day weekend and $\text{irl} > \text{blog}$. To summarize: Imagine we have some black-box environment with a state space $s$. At each timestep, we receive an observation $o$ (derived from $s$), and can pick…
I figure I’ll publish this long post in a couple parts, over today and tomorrow or perhaps the day after (as tomorrow is going to be a busy day). In today’s post I plan to go over curiosity and the Noisy TV Problem, to help build some practical and mathematical intuition. Tomorrow, I’d like to talk about some information theory, specifically aleatoric and epistemic uncertainty, and how this…
I had two assignments due today, and working on those consumed most of my time, so unfortunately today’s post will be rather short. One iconic quote from a classmate in my mathematics/philosophy class today ( 24.118 ): “So… mathematics is the art of making stuff up, and then justifying it with made up notation?” (TIL: There’s a whole branch of philosophy dedicated to determine whether numbers and…
“Take me and bind me to the crosspiece half way up the mast; bind me as I stand upright, with a bond so fast that I cannot possibly break away, and lash the rope’s ends to the mast itself. If I beg and pray you to set me free, then bind me more tightly still.” “I made by frowning to my men that they should set me free; but they quickened their stroke, and Eurylochus and Perimedes bound me with…
The other day I wrote a post titled On craft and AI that analogized AI to a hammer, a tool. I like the tool analogy because it does not hide accountability; it does not create an “accountability sink” . The actions of a tool are the responsibility of its handler. There’s a simple game called the Prisoner’s Dilemma where two people square off. Each player can choose to “cooperate” or to “defect”.…
I was curious to see whether I could find cheap cross-country flights for this upcoming long weekend. Spoiler: guess. The problem with sites like Google Flights (gflights) is that the interface only offers prices at the scheduling granularity of a day. the difference between a red-eye and a flight the night before can be pretty huge. For short trips, this granularity makes it hard to find flights…
Today is Sunday! What a great day. Today was absolutely freezing, and I just about froze my ears off while walking around. Luckily my friend Nathan saw me and spared me a beanie. This week was good! I wanted to start writing daily and this week, I was able to hit my goal! Not counting today, since last Sunday, I’ve written 7,957 words. This post puts me over the 8 kiloword mark for the month. My…
tl;dr: Today I typed up the first ~13-pages of a WIP specification for Isocore . The spec is about halfway done. I wrote it to be fairly approachable. Isocore is a distributed runtime for local-first applications. (Think of it like a BEAM-like runtime for untrusted Wasm Components.) I want to write an alternative to the web, and this is the first step. (Isocore is still under heavy development.)…
Games, Rules, Laws ? Writing is a game. A while back I came across the best linguistics paper . Anne Cutler wrote it, line by line: The Perception of Rhythm in Language . Today of all days is a busy one, so I dug up some old writing. Enjoy these lines from an earlier time, a collection of rhythmic phrases. The best authors of prose are those who would rather be poets. Read a poem disguised as…
Yesterday I wrote ~2,500 words, so today will be a little shorter. A couple weeks ago I started working on a now ~5k word essay that has unfortunately become three essays in one. Out of laziness, I’ve pulled one thread out of that essay, and abridged it here. I hope you enjoy. In Seduction , Jean Baudrillard talks about the difference between Laws and Rules . Laws are those fundamental immutable…
In 2016, I stumbled across otoro.net , which is still the gem now that it was then. The website is full of funky little neural networks doing all sorts of amazing stuff. And I love it. I started programming when I was 9, in fourth grade. When I was a kid, I wanted to be a pilot or an illustrator. Like for children’s books. Maybe an aerospace engineer if flight school didn’t work out; but I didn’t…
A quick post tonight because I should already be in bed. A lower-bound estimate for the number of calories the average adult American expends exercising a day is about ~150 calories per day (spent working out over the course of 30 minutes). According to the US Census Bureau, there are around 200 million Americans between the ages of 18 and 65. Multiplying these numbers, American adults expend…
Whenever you’re at a transition point in your life, I wonder why I’m writing this! moving to a new city, starting a new semester, changing jobs, making a life decision—you need to pay close attention to habits: both the ones you desire to keep and the ones you wish to lose. Some habits are reinforced by your environment; others are encouraged by those around you. I like to think that habits don’t…
Today is Sunday, the day of rest. My day today was very good. (My goodness it’s so nice to be back home in Cambridge, MA.) I was debating whether or not to post anything today, but I figured that as I’ve been going strong for 2 days, I might as well keep it up. N.B. If you read this blog over RSS, and you’re seeing all these posts and are worried it might be a little much, please let me know and…
I’ve been a fan of Conflict-Free replicated datatypes for a while. I wrote an old blog post about implementing them back in 2021 , Wow, almost 5 years ago. and I’ve been turning them over in my mind ever since. If you’re not familiar with what a CRDT is, I’ll provide this explanation I wrote, taken from a project I am currently working on: The project is called Together, as a part of Veritable /…
“Well okay I’m so glad you got to call him, who was he? The Dean of AI at Google?” “Haha his name is Dean. Jeff Dean. But yes.” — Conversation with my girlfriend after the call. I had the opportunity to meet Jeff Dean today, on very short notice. I am at a loss for words. He is an awesome guy. I never was into pop stars or celebrities as a kid. (I know, crazy.) I only realize now that, although I…
Happy Sunday! Two days ago, I woke up at 5:45am to row 26.2 miles. I walked on to the rowing team at MIT a year and a half ago. That might have been the greatest decision for personal development I have made. The row was hard work but I enjoyed it. I really enjoy rowing. When you’re working hard there’s this acute pain and slow fatigue, but you learn to push through it and the feeling becomes…
What’s the difference between a library and a framework? It depends on your definitions. Here are mine: When using a library, you are in control : a library provides a collection of behaviors you can choose to call. When using a framework, the framework is in control : a framework chooses to call a collection of behaviors you provide. As a programmer, I prefer using libraries. It is nice to be in…
tl;dr: I trained a neural network (NN), with logic gates in the place of activation functions, to learn a 3×3 kernel function for Conway’s Game of Life. I wanted to see if I could speed up inference by extracting the learned logic circuit from the NN. So, I wrote some code to extract and compile the extracted logic circuit to bit-parallel C (with some optimizations to remove gates that don’t…
N.B. I’m publishing some gems from writing I did over the course of two years while living in Brazil. Read the first post for more context. 2022-12-03 · Isaac Clayton · 87 minutes spent One problem I often run into when I start writing is that I have either too few ideas or too many . To put an idea on a page, we have to isolate it, find its form, and pin it down. Unfortunately, ideas are slippery…
N.B. I’m publishing some gems from writing I did over the course of two years while living in Brazil. Read the first post for more context. 2023-09-13 · Isaac Clayton You’re reading this in Chrome. Or, probably Firefox. Or Safari. Heaven-forbid Edge (only joking). There is a web browser hegemony. More than 60% of the world uses Chrome . As we’ve seen with recent proposals for Web Integrity ,…
An old note. I’m publishing a backlog of writing I’ve done. Read the first post for more context. 2024-03-26 · Isaac Clayton I noticed something while playing the piano a while back that I thought was pretty interesting. It has to do with how I practice and remember things. I play piano mostly by memorization. I can sorta read sheet music: I can read and play with one hand just fine, either left…
I’m publishing a backlog of writing I’ve done. Read the first post for more context. On today’s menu: I have been thinking about CRDTs and synchronization for a few years (e.g. this old post about CRDTs). I’m a fan also of functional-relational programming (but think in its purest form it can look so much cooler than React). In the post that follows, I try to pull some of these disparate threads…
Context I wrote this piece—like, let’s see, gosh where did I put it… ah yes here it is—on 2022-11-26 . That’s 870 days ago. For the non-walking calculators (non-calculating walkers?) among us, that’s 2 years 4 months ago. I lived in Brazil for 2 years. In Rio Grande do Sul , what a wonderful beautiful place! Living in Brazil was a great experience, perhaps a story for another time. (One day I’ll…
Managing side effects in pure functional programming languages has always been something of a challenge. Functions in purely functional languages produce outputs solely dependent on their inputs, by definition. Purity makes it easy to reason about functions: because all context is explicit, functions also become easy to break apart and refactor. The issue, however, with explicit context is that it…
DeepSeek-R1-Zero is cool . I wrote about reasoning models before o1, and I’m excited to the way this area of research has been cracked wide open, it seems. It’s also remarkably simple. I’m messing around with llama (running locally!), trying to see if I can at least partially reproduce the results (for fun). I figure I can collect reasoning chains and then adapt some existing RLHF code to…
Stop showing me spinners, show me what’s loading. Recently, while on a trip between cities, we hit a patch of road without good coverage. While on long trips, I enjoy reading books I have saved on my phone. I have some books on my phone that exist as PDFs stored on Google Drive. Before the trip, I started reading a few books while online, and I know they’re downloaded. Even so, while trying to…
Merry Christmas! Back from the mission, first semester at MIT is in the books! Now I am at home, with family, on a break from school. A couple days ago, I was telling my younger brother how cool Zig (the programming language) was. He was like, “if Zig is so cool, why don’t you … like, use it?” Oof. So I installed Zig , pulled in some neat bindings for raylib , and spent the afternoon writing a…
Note: This post was published before the release of o1 on 2024-9-12. People are quick to point out that Large Language Models (LLMs) tend to hallucinate facts and lack the ability to reason. LLMs are not grounded in reality. Hallucination is an architectural limitation due to how Transformers, as auto-regressive sequence predictors, are constructed.
Or, a GhostCell Deep Dive . In this two-part series, we build GhostCell from first principles. In Part I we go over the underlying theory required to understand GhostCell. This post’s been sitting on my hard drive for about 9 months, so I’ve decided to bite the bullet, split what I’ve written in two, and hit publish on Part I . Hope you enjoy! Over the past month or so, something I’ve repeatedly…
What really helped me begin to grok how Rust really worked was working towards building an understanding of how its compiler works.
The Beginning of the End (of Today) As I write this today, it’s tomorrow. And… We’re talking about school? N.B. This post is not about school, bear with me. Throughout the school year, I’m usually pretty consistent with my sleep schedule. It’s like a rhythm, every hour of my day planned out. I wake up at 5:55 exactly, do some morning studying (it’s always so much easier to get stuff done in the…
The joy of writing a new programming language is coming up with novel ideas and seeing if they stick. The challenge I’m attempting to solve stems from dealing with different types of objects that share common structure or behavior. For this reason I’ve been thinking a lot about how to rectify open/closed enumerations, traits, and type constructors. Traditional object-oriented languages deal with…
Preface Since September of last year, I’ve been chipping away at a long-form post on how to take good notes. This, sadly, is not that post—consider this post a teaser to tide you over until then. Today I finished my fourth book of sketchnotes. Each book has about 240 blank A5 pages, so I guess I’m fast-approaching the thousand-page mark. In celebration of filling yet another volume, I took a trip…
The other day, I was thinking about this question: What’s something that is not common today that you think will be really common 5 years from now? As I do a lot of programming language design, I can’t help but think about what our tooling will be like in 5 years. I’m writing this in 2022, so here are some of my predictions for 2027, in no particular order: Copilot & co. First things first,…
Note: This is a quick piece that assumes some prior knowledge of PoW and backpressure. If you want to build up an intuition about backpressure before jumping in, you could read this piece or that one . If you’d like to learn more about PoW in the context of this article, check out this article (and that one as well ). On the bus this afternoon, I was reading up on backpressure in distributed…
A Conflict-Free Replicated Datatype is a bit like a smoothie: the same ingredients will produce the same result, regardless of the order in which they are added. In the context of, say, text editing in a distributed context, merging two documents will always succeed in a deterministic manner. In other words, A CRDT is a bit like a git repository that never has merge conflicts. There are many…
Note this is a collection of a set of discord messages sent on the NixOS discord server. Troubles w/ NixOS I’m back! This is a hard one, I swear, and I’ve been banging my head against it for hours: I’m trying to compile a rust library that wraps libcec . To wrap libc , this library uses bindgen . Here’s what the crate roughly looks like: . ├── build.rs ├── Cargo.toml └── src ├── lib.rs └──…
#floating-nav-img { vertical-align: middle; display: inline-block; width: 30pt; height: 30pt; border-radius: 40pt; margin: 0; padding: 0; } #floating-nav { background: var(--fill-bg); border-radius: 40pt; position: fixed; z-index: 999; top: 20pt; right: 20pt; border: 2px solid var(--fill-bg); } #floating-nav > .tag { display: none } #floating-nav:hover > .tag { display: initial } Isaac Clayton ·…
#floating-nav-img { vertical-align: middle; display: inline-block; width: 30pt; height: 30pt; border-radius: 40pt; margin: 0; padding: 0; } #floating-nav { background: var(--fill-bg); border-radius: 40pt; position: fixed; z-index: 999; top: 20pt; right: 20pt; border: 2px solid var(--fill-bg); } #floating-nav > .tag { display: none } #floating-nav:hover > .tag { display: initial } Isaac Clayton ·…
Passerine is at an interesting point: we’ve established a few language features, and built this easily extensible functional core on which to base the rest of the language. We currently have two implementations of the language, one written in Rust, the other in D, and it’s imperative we set the course of the language before divergence occurs.
This is a frozen mirror of notes from Cornell’s CS3110 . Full disclosure, I did not write this. Topics Type inference and reconstruction Constraint collection Constraint solving (unification) Intro Java and OCaml are statically typed languages, meaning every binding has a type that is determined at compile time —that is, before any part of the program is executed. The type-checker is a…
Hello! This is my first blog post. // this is a function pub fn main () { println!( " Hello! " ) } I’m debating whether to keep this up… oh well ;P
Red, Yellow, Blue. These are the oft-repeated primary colors, but they are only a half-truth. There are really two sets of primary colors; which one to use depends on the type of light you’re working with. For additive light, like that emitted by screens, the primary colors are Red, Green, and Blue — RGB. The physical medium, on the other hand, is subtractive. Adding colored pigment to paper…
Note This piece is a work in progress. An interesting design space in the field of programming language design is that of memory management. In short, programs produce data while running. This in of itself isn’t much of a problem: in fact, it’s a good thing! If your programming language doesn’t allow for the production of any useful data, you might want to take a closer look at it… As time goes…
Note This post was written early on in Passerine’s development, before we had an FFI to interact with Rust. The FFI is still not fully finalized; most notably, as of writing this header, we need to still implement serde-style macros to easily interface with Rust datatypes, implement a module system to allow for external modules, and (eventually) allow for FFI interaction w/ Wasm modules for…
