On-Demand Software

As a forewarning, this piece may be a bit meandering, as it’s an assemblage of some previous things I never posted but that felt relevant to assemble now in light of the Anthropic blog post on getting Claude Opus 4.6 to write a C compiler.

A lot of ink (okay well HN comments) has been spilled over discussing the model’s ability to do this, and if its approach was valid, and if it was actually “doing” anything besides cribbing from gcc from its actual model weights.

I think these are all valid criticisms, and would even add that I wouldn’t be surprised if older models were able to do what that blog post demonstrated. It seems the key innovation of display was the new “agent teams” feature, which is not wrong to describe as a Sherlocking of the tmux/git-worktree workflow that was the multi-agent pattern de jour for the last 6 months. I would bet that Sonnet 4.5 with the git-worktree/tmux setup could probably do the compiler as well. Maybe.

Regardless, what a lot of the discussion has focused on was the fact that the end goal was reached after iteration. The model made some initial output, then the human incrementally steered it with docs, pointing to tests, etc. The model eventually got there, and we are all discussing this output.

But I actually want to reframe the discussion a bit, because one thing I think doesn’t get discussed enough is what that first artifact was. That first output before the human started to iterate. Whereas I think you could get Sonnet to eventually make a C compiler, I’m almost certain the first version of the compiler from Opus 4.6 was better than Sonnet 4.5, which would be better than 4, 3.5… etc.

Right now in AI development, a lot of the discussion is framed around iteration. Context engineering, prompt engineering, etc. all is often discussed in terms of initial setup, yes, but also ongoing iteration. Part of the reason for this is that AIs seem exceptionally bad at this, which I think is both an AI problem as well as a Human problem. Regardless we try to find solutions but it’s clear here that it’s “lossy” — it’s not like after a single prompt following an AI’s first pass do we get the perfect result, but instead that it often takes a lot (though less than before) of time.

But what I want to think more about is that first output, the one-shot. That’s clearly getting better. In fact, I think the one-shot is getting better at a rate faster than we are able to figure out new tools for iteration. Skills, MCP, Subagents, etc. are all basically hacks stacked on top of the base model, abstracting away the idea of prompting but ultimately all boiling down to different flavors of prompts.

Taken over the course of a year or two or ten, it seems apparent that AI’s ability to one-shot software will only get better, provided the models themselves get better (which seems it will be the case). And of course it’s worth saying, if your one-shot isn’t great the first time, prompt another one. Or 100 of them.

What does that mean, when any software can effectively be summoned in a near-perfect fashion on demand? Anthropic has actually pitched an idea similar to this in a seemingly forgotten research project that I think is maybe one of the most interesting things they’ve done, but I’m thinking more general here.

You imagine some software, and then it exists seconds later. It’s custom, tailored exactly to your needs, and you also spent a nominal enough amount of money on it that you feel totally fine discarding it immediately after use.

What isn’t happening here, between an idea and a software product? Well for one there is no programmer necessarily. And there is no human-authored programming happening.

Which is interesting. Because so much of the whole idea of software has been tied to the idea of programming being a thing humans have to do, and the whole of programming as a discipline has developed around the idea of humans being the ones that have to program. Programming languages are for humans.

I just want to say that again:

Programming languages are for humans.

They are abstractions used to ultimately describe how electricity should move across silicon.

This is also coming to mind based on my day to day work at Unity on the IL2CPP team. It’s not a secret what IL2CPP does — it’s turns IL code from C# into C++ code. That C++ is then compiled for a target platform. This is all fine, but the thing is that the C++ code it writes isn’t really C++ a human would ever really want to work with. In fact, it’s not even really doing many C++-y things. It’s basically “very dumb” C++ code that more or less behaves like C.

Why? Well, we have compilers. If we instead wrote highly abstracted C++ code, it would be harder for a compiler to optimize that code. So we produce very dumb code and lean into the affordances of clang/gcc/MSVC to make the code itself fast by inlining functions, using vectorized instruction, etc. We could always do better here obviously, but the point is that we are generating code meant to be compiled, not really used by humans.

We could just as easily be generating Typescript. Or Haskell, or whatever. The language choice is due to compiler dependencies for other platforms. The programming language is being chosen on the merits of it’s ability to act almost like an API more than for some reason about the language itself.

Which is notably very much not how we typically think about programming languages. I think there is some discussion of “compatibility” broadly, but the choice of a language is often much more about personal preference or domain-convention. But things can obviously move over. You can use Clojure to make a game. You can make a website in C, etc.

LLMs take this idea to the extreme. LLMs writing programs with programming languages is Weird. It’s weird we are having them generate an abstraction that is an affordance for humans, when humans (increasingly) don’t so much care about that abstraction - why not get them to start doing things like generating binaries directly? Or just ad-hoc machine code?

Part of this is a limitation of computing hardware, and doing things on a computer requires an abstraction and tooling to ultimately do something conceptually very simple. You can’t just execute a mov instruction on a machine… for now.

But maybe they can? At some point?

Assembly itself (assuming we need it), is also very token inefficient. Even trivial programs can be large amounts of assembly. Maybe context windows get large enough that this matters less? But going to assembly even feels like a hack. The idea is not to “get to a lower level language”, but more something about skipping programming languages as the interface altogether. Going from idea to thing with no intermediate steps (besides inference).

Some of the things that got me thinking about this are from the post here: https://alperenkeles.com/posts/llms-could-be-but-shouldnt-be-compilers/

If you say “give me a note-taking app,” you’re not describing one program, you’re describing a huge space of programs.

The interface we are using largely for LLMS is that we talk to them with human language that gets compiled into programming languages which get turned into programs. And it just feels like we don’t need that middle step, we should be able to infer programs directly.

If this is the case, what is the interface between english and inference? Programming langauges are not well suited to this — programming languages describe something very specific, and instead we need some sort of representation that is able to capture the idea of a decision space.

Does it look like a big planning doc? Or is it something like a “visible” AST node graph thing that you then prune or branch? Do ideas exist as leaf nodes? How do you “write” new things here?

You can imagine a web of nodes, 1000s of gitworktree-ish programs that are implied by your prompt, each made perfect by inner agent loops working at 1000x the speed of tools now. How do you move through that? Is it langauge? Is it text?

Or is it some intermediary IL-ish language for LLMS like Memelang, some weird mix of tagging concepts and logic programming and python. A language that doesn’t actually need to be “compiled” but smells very similar to a programming language and gives the LLM some idea of determinism.

My point is that, what that intermediary needs to be doesn’t have to be a “programming language” as we think of it now. It probably will be for a while, especially as long as we have compilers. But maybe for not as long as we may think. The idea of us having to abstract the idea of “talking” to a computer may be waning.

Noahpinion wrote about this a bit here:

A generation of smart apes trained themselves to make their brains act like computers; it’s hardly surprising that computers eventually rose up and reclaimed their core competence.

Whatever this “new thing” looks like feels like it won’t be “programming”. Calling back to the previous idea of “the possibility space of all programs that could describe an idea”, working with this new thing will likely be about better honing that possibility space. Some ability to explore the space of designs implied by your initial prompt.

And to not forget what I was saying earlier — we will be able to rapidly generate a lot of these very quickly very soon. Maybe the “programming language” is less the language and more that a prompt automatically generate 10000 binaries immediately and you can figure out what you want. And then also discard it when you’re done. “Programs” don’t need to be long-living.

I think especially with things like world models, this type of thing starts to come into focus a bit more. There is no “compiling” here — there is a prompt and there is output. We already get this in chat-based models, but this seems around the corner for “programs”. I don’t even know if we will call these things “programs”? What if they embed their own prompted logic, almost like a rig for a 3D model. You tweak values of the inner workings of something and new programs are produced (not just new behavior).

It’s an exciting prospect, but also obviously a scary one. I don’t know what the future of programming is. I think taste matters, if you’re making something for someone else (or lots of people). But I think a key change here is also that software can be on-demand and disposable. Willing a program into being only for the duration of its usefulness means that a lot of people will have fully custom software built only for them, for small (or long) durations of time, where no “programmer” is in the loop for any part of that.

That’s already sort of happening, and I expect it to continue happening more and more. We’ll see what’s next.