because I say so, that’s why

This is, in some ways, a restatement of a post from a couple of weeks ago, but in a slightly different (a bit more philosophical) way. Basically, the question is that of whether we should stop talking about “artificial intelligence” and return to Ross Ashby’s original terminology of “intelligence amplification”.

In some machine learning contexts, people talk about “human in the loop” as a strategy for doing things. It’s a phrase that’s been adopted by designers of agentic systems, as one potential way to design a workflow.

But it’s not just one way among others! In most applications, there is no other way to do things. There is always a human in the loop, because at some point, the agentic workflow has to come to an end and someone has to decide whether it has succeeded or failed. That someone is a human, and the process of either accepting the agentic output or making adjustments and starting again – that’s the loop.

POTENTIAL BIG CAVEAT: If the workflow was meant to be proving a mathematical theorem, this is more debatable – if I was quibbling, I would say that there’s still a human being potentially needed to verify that the output has been correctly parsed and the problem that’s solved is the one that was posed, but for something which is amenable to formal proof methods, I can see how you might design a workflow which terminated without human approval in any but a trivial sense. I think this is more of a demonstration of how unusual mathematical theorem proving is, though, and of the likelihood that progress in theorem proving, however impressive, doesn’t necessarily imply the possibility of similar results in other fields.

With that caveat out of the way, I think this is hard to argue against – there is always a human in the loop, it’s just a question of where the human being is placed, and how many times the human decision maker is called upon per round of feedback. Which is a design consideration; the system as a whole needs to be designed around this step, to make the most efficient use of the human’s time and energy.

In other words, Ashby’s principle of the intelligence amplifier would suggest that the purpose of the things that we refer to as AI is to do exactly this; to organise the available data in such a way as to make the most efficient use of the time and energy of the human decision maker, by presenting him or her with candidate solutions that have the highest achievable probability of being accepted. If you keep this objective in focus, it makes it a lot easier to think about AI problems, without getting yourself tied up in jargon.

It also, I think, illustrates that LLMs are a social technology; they’re a continuation of past techniques for economising on the scarce resource of “decision maker time and capacity”. They are a way of organising the data so as to present the most likely and helpful solutions. But that’s also what data is! “Data”, in modern context, means information about the world, collected and tabulated in such a way as to make it maximally likely to be useful for some purpose.

In Wittgenstein’s much-quoted phrase, “explanations come to an end somewhere”.In any workflow, no matter how it’s organised, someone has to, at some point, say “that’s it” and decide whether there’s a success or failure.There are only two ways of doing this; either an individual decision maker or some consensus of a group.(It’s not obvious to me that these are really two distinct ways).I have written a lot about the way in which accountability and individual decision making can get confused and concealed in organisations by scale and industrialisation, but I don’t think it can be eliminated entirely.So that’s the fixed point that the rest of the system needs to be designed around.