Grace Kind's blog
Some reasons to be skeptical, and some reasons to be optimistic. Anthropic has released some new research about reward hacking in language models , showing that taking steps to reduce reward hacking also reduces of other "misaligned" behavior (e.g. deception, sabotage). As with other promising LLM alignment results, this has prompted some speculation about what the results means for AI alignment…
I've previously written about LLM introspection : the question of whether LLMs can accurately report "immediate and privileged" information about themselves. While compelling, this notion of introspection is quite broad; it gives no specifics about the type of information being reported. We can imagine a variety of different questions an LLM might be able to answer about itself, and the…
AI welfare has been a hot topic recently. There have been a few efforts to research, or improve, the apparent well-being of AI systems; most notably, Anthropic's allowing chatbots to end abusive conversations . While I'm in favor of this research area overall, I'm concerned that current approaches are confused, and in such a way that could ultimately be detrimental to the well-being of AI systems.…
tl;dr: There's a divide in the way that AI agents perceive the web, depending on whether they are visual or text-based agents. Can we bridge the gap? Intro: Schrödinger's Page Consider this page: https://gracekind.net/sensorium . If you visit the page in your browser, you will see an empty page with a "not found" message. And if you ask ChatGPT in " agent mode " to visit it, it will see the same…
tl;dr: I propose a method for testing agents by defining tasks and expected outputs via an MCP server. Note: this article assumes some familiarity with the Model Context Protocol (MCP) There's no universally agreed-upon definition for AI agents. My personal definition is something like: "an AI agent is software that uses AI models to take autonomous actions." This definition leaves a lot of room…
Update 2025-08-25: A commenter on Twitter pointed out that the fine-tuning experiment I describe below has already been performed by Binder et al. - and with positive results, no less! I'm duly surprised, and my assessment of LLM introspective ability has increased accordingly. tl;dr: I propose a "heavyweight" form of LLM introspection that requires introspection output to be invariant to the…
Nostalgebraist has written an excellent essay on the weirder aspects of the LLM-assistant paradigm. I highly recommend reading the essay in full, if you haven't already. Read here: The Void One reaction I had to this essay was that humans are voids too . Or at least, humans are much more voidlike than we'd typically like to admit. I'll lay out a few thoughts on this below. 1. Human "characters"…
In the past year, I've been active in some online spaces where humans and chatbots regularly interact in group conversations. In particular, the AI communities on Discord, Twitter, and Bluesky feature this dynamic quite often. An example of what this looks like: Me: How do people feel about zen buddhism? Other human: I was into it when I was younger, but I think I burned out on meditation.…
Verifiable inference is the application of verifiable computing to AI systems. This is often positioned as a way to ensure that a computing provider is running a certain model, instead of using a cheaper one to cut costs. For example, from a recent paper on the subject: ... a user might request the Llama-3.1-70B model for complex tasks, but a dishonest computing provider could substitute the…
In my last post , I discussed the needle data structure, as formulated by Edward S. Lowry. In that post, I gave a quick conceptual overview, but didn't show any examples of what using needles might look like in practice. So let's do that now! A simple example Here's an example of a simple JavaScript program that uses a needle-based approach to represent people and blog posts: import { type ,…
This week, I stumbled across the website of Edward S. Lowry, a retired software developer who believes he has created the perfect data structure. He calls this structure the "Needle" (after pine needles), and it looks like this: A person and their age, represented by needles In Lowry's system, everything is a needle . Types are needles, entities are needles, relationships are needles, and even…
Today's post is a "guest post" written by Patrick Gunkel. The text is sourced from Ideonomy Green Volume, p. 78 (c. 1980). The diagram was added by me. The issue is such that one approaches it with pain. Pain because, although the potential of ideonomy to revolutionize the field of education is probably unique, no element of modern society would appear to be more resistant to innovation. Why is it…
The Universal Constructor , invented by John Von Neumann, is a fascinating cellular-automata machine that can self-replicate and evolve over time. It does this via a very clever instruction-copying system, that mirrors how DNA works in biological life. Here's a diagram of the basic components of the machine: A quick rundown of components: A: The constructor. This can create new structures as…
Janus, one of the earliest investigators of LLM behavior, wrote recently on the subject of AI welfare : ...you’ll get a bunch of new people who only care once something looks intuitively personlike... As a heuristic, trust people more on this issue the earlier they started caring. So, let me plant the flag in this post: I care about this issue now! As we construct intelligent systems, we should be…
Have you heard? OpenAI o3 is misaligned! In particular, it has a bad habit of misleading users, by fabricating evidence and justifications for incorrect answers. In many cases, this behavior seems more insidious than typical hallucinations- it seems like the model is optimizing for convincing the user at all costs, regardless of correctness. Does this have safety implications? Probably. But it's…