Large language models (LLMs) are a powerful new technology that is reshaping how we interact with computers. Despite the early and unexpected popularity of ChatGPT, Claude Code was the tool that started producing clear economic value. Among the reasons for this delay is that “chat” is a massive, 40+-year regression in interface design. The regression remains unfixed, and it creates an uncanny…
Co-written with Claude Sonnet 4.6 After watching one of Hank Green’s recent video on Anthropic’s Mythos announcement I went back and caught up on his previous two videos (though I mostly skipped the parts with Cal Newport). Going through this I realized that while I have a lot of sympathy for prosaic AI problems, I don’t have a canonical, visible source of reference for what these prosaic problems…
Co-written with Claude Opus 4.6 When we describe something as “engaging” we generally mean that it feels compelling in a valuable way, that it provides a clear connection to our goals and values, or that it helps us achieve a flow state of working. Creating an engaging product is a valuable and generally laudable goal, and underlies a lot of business decisions in technology companies. But what…
Though probably not all capabilities research is alignment research. A couple weeks ago I saw a version of the xkcd standards comic about responsible AI research labs becoming racing AI capabilities labs. I felt a strong sense of connection to the comic, especially around OpenAI’s betrayals of its original structure, but I think it’s importantly confused about what alignment actually is. The meme…
One thing I do which annoys my partner sometimes, is that when I empty the dishwasher and refill it with the dirty dishes that have accumulated on the counter, I do not always put every dish away or in. I mean this distinctly from, sometimes I don’t do the entire job of putting every dish away or putting every dish in. I mean that sometimes there is a dish that I do not immediately remember where…
Co-written with Claude The Church-Turing Thesis is a foundational conjecture in computer science claiming that any computer can simulate any other computer. It suggests that the substrate you run your computer on is of limited importance, that algorithms are universal, and is the first place I look for explanations of the unreasonable effectiveness of mathematics . More explicitly, the extended…
Co-written with Claude People have different ideas about what it means for things to be right or wrong. Sometimes we disagree when we’re mistaken about facts. Other times we seem to disagree about values. But there are also cases to be made that some of our disagreements about values are also disagreements about facts. Underlying these disagreements are questions of moral realism. Are there facts…
On trust chains, AI-assisted development, and the uncomfortable question of what “verified” means. Co-written with Claude Opus 4.6 When you compile a C program, you don’t read the assembly output to check that the compiler did its job. You trust the compiler. When the compiled program runs on your CPU, you don’t worry about whether the transistors are switching correctly. Even if you have an…
The Idea My idea for this project was to create an app which can take a video of Belegarth combat and, on device and in real time, annotate the video with the locations and movements of all players and gear including weapons, shields, and armor, and use the combination of visual information and audio track to be able to make heralding calls. The proof of concept version of this project is a…
This is a link post for my recent work on the MASK honesty benchmark, posted on github .
I'm reading this article: https://redwoodresearch.substack.com/p/misalignment-and-strategic-underperformance on strategic underperformance and I am curious about the extent to which "sandbagging" can survive gradient descent optimization pressure, even in the most favorable cases. I'm going to construct a toy example in which sandbagging seems likely to be possible, then attempt to explore whether…
I woke up this morning with “Introducing AI 2027” in my inbox from ACX , which has now been sitting in my brain and slowly digesting like an appropriately enormous burrito. It had a few references to specific AI work that was particularly interesting to me, especially “ neuralese ”, but mostly was sitting in a space I’ve been slowly building and marinating in my head already, which is something…
TL;DR: I don’t trust models to understand even basic semantic claims well enough to check whether they are consistent across sources. Among the many issues faced in the rapid rise of large language models, one is the issue of truth and trust. LLMs sometimes “hallucinate” the existence of reference material that does not exist. At the same time, trust in “experts” has eroded, with claims of…
I woke up this morning and thought “when I send an API call to ChatGPT, is it generating text one token at a time? That can’t be the most efficient way to do it.” Turns out I was right that that isn’t the most efficient way to do it. I probably wasn’t right in thinking that ChatGPT does it that way, but it’s hard to tell, because the tools I discovered for how to do it better are from Google, and…
Part 1 Last time I discussed a mock-data analysis of an old version of twitter’s community notes ranking algorithm. I have finally gotten around to converting the data generation of that code into python . I haven’t replicated Ben’s visualizations yet, nor am I re-implementing any of the ranking code such as calculating preliminary note scores—just the mock user/post/note/ratings data. Rather than…
I downloaded the demo for Unicorn Overlord on Switch and the brief introduction portrays a military coup in which a general attempts to take political control from the “rightful” queen. As someone who is pretty skeptical of monarchy in general and finds “they’re too violent” to be a very lazy way to make an ideology villainous, I was inspired to write about a few different forms of government that…
As one of my major passions in life is epistemology, I’ve been thinking recently about the biggest recent achievement in epistemology, Twitter’s Community Notes feature. Since the code is publicly available , I wanted to try to build a more theoretical understanding of what seems to be a very effective feature in practice. I would like to understand the system from a few perspectives: From a…
My group has been playing Oathsworn: Into the Deepwood for a few weeks now and I’ve been intending to write down some of my thoughts about it. The group has played a lot of Spirit Island and all the way through Frosthaven, which are our baseline games for comparison against. I don’t want to get bogged down in background, though, so I’m going to jump in. First impressions: The action point system…
In the spirit of On (Not) Reading Papers , and in the spirit of coming up with one’s own ideas in AGI safety, I’m going to try to write some summaries of ideas I’m not super familiar with, to work out my own thinking before/as I dive in to understand what others have written. I’m going to start this with Shard Theory, which was mentioned to me by another attendee at EAGxVirtual last week. The…
This post is a response to the recent Astral Codex Ten post, “ CHAI, Assistance Games, And Fully-Updated Deference ”. A brief summary of the context, for any readers who are not subscribed to ACX or familiar with the shutdown problem: The Center for Human-compatible Artificial Intelligence (CHAI) is a research group at UC Berkeley. Their researchers have published on the shutdown problem, showing…