Returning to the so-called “AI” thing, here, because this nonsense shows no sign of going away, and it damn well needs to. As many other sensible people are pointing out, thankfully, including James Fodor in this excellent article here.

I was moved, a while back, to lay out my own credentials on the topic, over on File 770. I studied Artificial Intelligence for two years as an “outside subject” – part of my Linguistics course at the University of Edinburgh – and went on to the field of computational linguistics, or how computers could help with things like speech recognition and automated translation (it was the early 1980s, so the answer was basically “not very well”). Later on, there was my infamous failed research project, which involved a very recondite sort of machine translation indeed. So, is my knowledge of the subject incomplete, superficial, and decades out of date? Yes. Do I still reckon I know more about it than Elon Musk and Mark Zuckerberg put together? Also yes.

The history of AI and robotics could be neatly summarized as “it turns out that’s more complicated than we thought.” In the good old days of SF, it all seemed simple enough; you would have an electronic computer that would serve as a brain, video cameras for eyes, microphones for ears, and some sort of humanoid chassis that would move about, and hey presto, you would have a humanoid robot. Just needs some of the details working out, that’s all. Obviously, your first lot of robots would only be capable of simple functions, like walking, talking, fetching and carrying, but once those marvellous computer brains were improved on, you would have robots capable of anything people could do, only faster, and tirelessly, and with absolute mechanical precision.

Guess what, it turned out to be more complicated than they thought. The basic human functions such as walking, seeing, and understanding speech are actually extremely complicated indeed; we take them for granted because we have been honed by millions of years of evolution to do all these things as a matter of course. The head of the AI department while I was studying it was heavily into visual processing, so we hapless students got a lot of information about the incredibly complex mechanisms of the optic nerve and the visual cortex that enable us humans to, well, see. Soldering a video camera to a motherboard just isn’t going to cut it.

It really is important to remember that the whole “the brain is a computer” thing is what we literary types call a metaphor. It’s a useful (usually) parallel, but it isn’t actually the case. It’s important to remember all the ways in which the brain is not like a computer at all. It’s not made of the same stuff, it’s not wired up the same way, it uses comparatively sluggish electro-chemical impulses instead of pure electricity. It is complicated and error-prone, but also error-tolerant to a degree impossible for electronic circuits. You may well have heard the story of Phineas Gage, which just goes to show how fault-tolerant the brain can be; try firing a six kilogram iron bar through a rack of servers sometime, I think you’ll find it doesn’t just make them sweary and grouchy. The brain adapts, learns, develops. And, rather importantly, it does so inside a human body, in which it comes, pre-packaged, automatically.

The body gets a bit of a bum rap, it seems to me. Whether it is Cartesian dualists talking about the mind/body interface, or Christians promising that we get to trade up to a better model like unto His glorious body, or transhumanists suggesting we ditch it and go for something in durable, wipe-clean silicon instead, there always seems to be this idea that the mind is what’s important, and the body is just holding it back. The mind is the true self, the essence of you, the thing which contemplates the infinite and appreciates beauty. The body is just that thing full of aches and pains that you shit with.

Better minds than mine (or more spiritual souls, or higher grades of meat-robot programming) have wrestled with the question of the exact nature of consciousness, and I’m not about to propound any answers to that one here. But, whatever our consciousness is, it is ineluctably entangled with the physicality of our bodies. All the experiences through which the brain grows and develops are modulated through the perceptions of our bodies.

This includes, by the way, a very definite sense of the self – a boundary where we leave off and the rest of the world begins. The sense of touch is constant and ubiquitous; we are always, through that sense, informed of things that are not us, but which are in contact with us. Also, we have a proprioceptive system, and while I’ve read about artificial ways of seeing, hearing, touching, and even tasting and smelling, I’ve not come across any references to that one in the AI literature. (Which doesn’t mean there aren’t any – my knowledge is incomplete and outdated, remember? – but the topic certainly takes a back seat to things like visual processing or speech recognition.) Basically, your brain gets constant feedback, from things called muscle spindles, on what every muscle in your body is doing, or having done to it, at any given moment. (Usually. I speak with some [lack of] feeling here, because my persistent ulnar neuropathy has left me with no proprioception in the little finger of my left hand, and even if you’ve never heard of proprioception, believe me, you miss it when it’s gone. I had a session with a physiotherapist a little while ago, and it was very, very weird to see him wiggle that finger about – I could feel him touching me, but the sensation of the motion was completely absent. No feedback, you see. Also, it plays hell with my typing. Ten-finger typing gets slowed down if you only know where nine of them are.)

There’s a cautionary tale I heard somewhere in my dubious education, about a student practicing with a CNC lathe and inadvertently mistyping a start parameter, which had the machine start cutting into a metal surface, with the start point half a centimetre inside the metal. The results were noisy and expensive. And if the machine had had a sense of touch, and/or a proprioceptive system, it would have picked up on the error.

Can a computer develop a sense of self, an identity, without that foundation of physicality? Can consciousness arise without identity? Well, I don’t know – as I said earlier, I’m not in a position to describe what consciousness is. But I’m pretty sure that these questions aren’t troubling the designers of large language models.

Speaking of which, let’s look at another one of those useful but misleading metaphors, the language of computers. Many of us have some notion of what computer programs look like; some of us are wearily familiar with various programming languages. Depending on the degree of abstraction and obfuscation involved, an ordinary human can follow and understand the instructions in a language like BASIC or Pascal; things like IF-THEN-ELSE or REPEAT UNTIL make sense to our human brains. However, these are the sorts of things that you find in high-level programming languages, and these are only abstract representations of the instructions that are actually going into a CPU. Programs in high-level languages have to be compiled or (key word here) interpreted into the binary code¹ the computer understands.

That binary code is interesting – to me, at least – because the act of putting that pattern of bits through the registers actually causes those bits to flip. It’s like what they call performative verbs – verbs where the act of saying them causes them to happen. “I promise” is perhaps the canonical example. (I wish some people would get it into their heads that “refute” is not performative – to refute something, you must present facts and arguments that disprove it. Just saying “I refute these allegations”, as so many politicians do, is not enough for an actual refutation to happen. I’m digressing again, sorry.) Similarly, you can regard the code – whose execution, once begun, cannot be denied – as a series of imperatives (the “verbs” in BASIC were actually called commands, after all.) So, when your program is compiled or interpreted, the vaguely-similar-to-English high-level language is being translated into another. Another language, that consists entirely of impersonal performative imperative verbs.

Those impersonal self-fulfilling commands affect the computer’s memory, so let’s have a look at that useful but (you guessed it) inaccurate metaphor. Stuff from the central processing unit of the computer comes and goes between it and the random access memory (RAM) chips, which are commonly compared to human short-term memory; every so often, information is transferred to the backing store (the hard disc, or SSD chips, or floppy discs if you’re old school, or cassette tape if you’re really old school), which is like human long-term memory. The operative word there, of course, is like.

The processor of a computer doesn’t actually remember what’s gone before, in the way that humans do. When 010111000111010110 pops up in a register (I am aware that computers have moved on past 16 bits, thank you, I’m just trying to save on the typing here), there is no sense of “ah, that reminds me of 00011110001101010110 and those long sunny afternoons drinking absinthe on the banks of the Seine.” That pattern of bits is just there, supplanting whatever pattern was there before it, and in a tiny fraction of a second it, too, will be gone, replaced by some other pattern. No computer has ever asked itself mais où sont les 0001110000110110s d’antan?

The contents of a computer’s CPU changes trillions of times a second, and there is no way of telling, from inside the CPU, whether its instructions come from the internal processes of its programming, or from short-term or long-term memory, or from an external sensor, or from some underpaid code monkey banging on a keyboard, or even from a solar flare or a defective microwave in the kitchen next door. Given this, it’s hard to see how a computer can, innately, have any continuity of experience, any conception of self, or any sense of the passage of time or the existence of space. Could technology progress to the point where it could have these things? Maybe. I’m pretty darn sure, though, that we don’t have that technology yet, nor is there any meaningful prospect of it on the horizon. Artificial Intelligence, if it exists, is not going to be even remotely similar to whatever kind of intelligence we have.

Incidentally, the notion of uploading human minds into a computer, so popular with the transhumanist mob, runs up against the same sorts of problems. We might be able to take a snapshot of the neural activity in a human brain (though I suspect it would take rather a lot of storage capacity²,) but running that image as a sort of executable within a computer, and having it do human-brain-like things, is going to take technical advances that just haven’t happened yet. The science fiction stories where you stick a fibre optic cable in your ear, flick a switch, and ZAM, there you are, standing on a circuit board dressed in Lycra with blue neon stripes, are going to remain science fiction. Some people sneer at the “primitives” who believed a human soul could be captured on a photographic image; I think the same derision should be aimed at the guys who believe you can capture one on a USB stick.

So, Artificial Intelligence, if it is achievable at all with current technology (and I suspect the technology which might make it achievable is some way in the future) will necessarily operate in a way which is radically, fundamentally different from human intelligence. So different that communicating with it, or even recognizing that it’s there, will present significant technological challenges. So why are we so happy – well, why are some of us so happy – to believe that Artificial Intelligence is with us here and now, ready to correct our grammar and do our homework for us?

I think the answer has a lot to do with the way the human brain works. Even the superficial level of study that I’ve had shows that our perception of the real world is very much mediated by the way the brain processes the input. There’s a whole lot of work involved between a shower of photons making chemical changes in your retina and your actual reading and understanding of these words. And the human brain, remember, is highly fault-tolerant. If some data is unclear or garbled, the brain will automatically fill in the gaps with its best guess at what should be there. Most of the time, it’s right – if it wasn’t, we’d all be walking into things and falling over all the time. Sometimes, it’s wrong. Sometimes, it’s predictably wrong, which is why you get all those stock optical illusions like the lines that look different even though they’re the same length. You get glitches where some people’s brains jump one way, and some another, hence (for example) the Dress That Broke The Internet. And that’s with direct perception – when we get into the realm of memory, well, that is subject to revisions and reinterpretations, not just to the point of forgetting things, but to the point of remembering things that never happened.

A few cases in point, from my own experience. First, let me tell you about the time I saw a UFO. I was walking downhill along a road in Gateshead, and my attention was drawn to something in the distance – a round, red, something, with a bright light flickering irregularly in the centre. It was darting in different directions above the hills on the horizon, suddenly stopping or zooming off in a new direction with apparent disregard for things like inertia. I was genuinely experiencing something like all those UFO “close encounters” that I’d read about and disbelieved over the years. Was this it? I wondered. Were our brethren from beyond the stars here at last, to impart their cosmic wisdom and welcome us into galactic society? And why had no one else noticed this thing?

So I walked on down the road a good hundred metres or so, watching this THING all the while, and suddenly something clicked in my brain, and I realized what it was. I mentioned the hills on the horizon, didn’t I? For some reason, my brain had glitched and believed that the object was as far away as the hills – but it wasn’t. It was much closer, and much smaller. It was a red balloon, with something reflective, possibly a bike reflector, stuck to one side. The apparently impossible motions – if it were a large object, far away – resolved themselves into the ordinary bobbing of a balloon in random air currents. The flickering light was just the reflector doing what it was supposed to do. And, once I’d realized that, I couldn’t see it any other way. My brain had retuned itself to the real world, and the fascinating anomaly had vanished. I won’t say I wasn’t a bit disappointed, but there we are.

Another example from a different sense, but again involving UFOs. This was a documentary about supposed alien abductees; one of them had recorded Sinister Sounds inside their home, a message from the alien visitors who were clearly planning to Do Things to them in their sleep. The recording sounded like a sort of intermittent hissing noise, but the abductee swore up and down that you could hear the words “don’t wake up” if you listened closely. And the documentary maker went back and played the recording again, and I could very plainly hear it saying don’t wake up… don’t wake up… don’t wake up… over and over again.

Some people might think this was proof positive of alien visitors/mind control/Masonic space lasers/³whatever. Being me instead of some people, I just thought, “oh, yeah, well, that’s the brain for you – the sound frequencies for the phrase are all there inside that random burst of noise, so once you’ve been told it is a phrase, your brain will automatically concentrate on those particular frequencies to interpret it as the phrase, you could probably make it say a whole bunch of three-syllable phrases if you wanted to.” It’s related to the phenomenon that phonologists call categorial perception – the human vocal tract can produce all sorts of weird and wonderful sounds, but any given language only uses a subset of them (called phonemes), so, if you hear a speech sound which isn’t in that subset, your brain tends to jam it into the nearest one that is a sound in your language. This is why (for example) English speakers pronounce “loch” as “lock” – the velar fricative represented by “ch” is a phoneme in Scots dialects and in Gaelic, but not in English, so it gets heard as the closest phoneme available, the velar stop represented by “k”. (You can learn to identify speech sounds that aren’t phonemic for you, but it does take effort – I remember lots of practice time with phonetics students in pairs, gazing intently into each others’ mouths.)

Memory? Well, I was going to offer a comic quote about witness statements, where a policeman says that according to the eye-witnesses, the crime was committed by “a tall dark short fair-haired bald man of medium height, with a hooked Roman nose, one blue eye, one brown eye, and one green eye.” Or something like that. I was sure I’d seen it in David Nobbs’s The Return of Reginald Perrin, so I got that book down from the shelf to verify it and check the exact wording. And I can’t find it. (The quote, that is, not the book.) And I think that just goes to prove my point, doesn’t it? Sometimes, human memory admits it’s faulty, and sometimes it tries to cover it up by linking in a whole bunch of material that looks like it might fit. People can, and do, sincerely believe they remember things which never happened, even in the face of compelling evidence that they didn’t happen. The brain is a wonderful thing, but it’s not always 100% reliable.

We think of ourselves as perceiving the real world, but our perceptions are mediated by our brains, which spot flaws in the input from our sense organs, and do their best to paper over the cracks. This means, among other things, that our brains are very good at spotting patterns and interpolating data to reconstruct things they’re familiar with. Unfortunately, this means our brains are also very good at spotting patterns which aren’t really there, which is how we get infallible systems for winning at roulette, conspiracy theories about Masonic space lasers, and the face on Mars.

That last one is important, because it accounts for much of the hype about “AI” – we are predisposed towards anthropomorphism, finding human-like qualities in things that aren’t human. You can attribute personalities and volition to inanimate objects based on the slightest of perceptual clues. A vacuum cleaner is just a vacuum cleaner, but Henry Hoover is a cheerful little chap, isn’t he? Stick a pair of googly eyes on them, and the most unlikely things become human.

Now, for most of humanity’s existence, the only source of human language has been human beings, so when you see or hear something producing human language, there is a very strong temptation to believe it’s human – even if it is a parrot, or a mynah bird, or something even less intelligent like a computer program. Computer programs do not have to be as sophisticated as Large Language Models for people to think they’re human.

“Ah,” says the knowledgeable reader, “he’s going to talk about ELIZA.” Yes, I am absolutely going to talk about ELIZA, because it is such a very good example. Created by Joseph Weizenbaum between 1964 and 1967, ELIZA is credited as the first chatbot; it was designed to emulate a psychotherapist, using a fairly limited set of stock responses which identified key words in its interlocutor’s messages and fitted them into templates for replies. And it had people convinced that it was a real person, that no mere machine could possibly understand them as well as ELIZA did. Now look at those dates again, and consider just how much technical progress there has been on the hardware end of things since then. ELIZA is not a sophisticated program. If you are the sort of weirdo who has a “smart” home, you probably have light bulbs with enough capacity to run ELIZA. Your fridge probably has enough computing power to run ELIZA and perform its normal fridgely duties (maintaining optimum temperatures and levels of energy usage, keeping an inventory of its contents and their expiry dates, and snitching on you to Amazon about your chocolate ice cream habit.)

So, since a rinky-dink little gizmo like ELIZA can successfully con people into believing it’s human, what chance do our poor gullible brains stand against modern technology? AI has genuinely come a long way since I staggered through the doors of Edinbugh’s Artificial Intelligence unit⁴; moreover, the hardware supporting it has progressed at a ferocious rate. A lot of seemingly intractable problems become rather more tractable if you can simply throw more processing power at them, which explains why things like text-to-speech, optical character recognition, speech recognition, and machine translation have reached the point of being, well, mostly adequate these days.

Mostly. Anyone who’s watched machine-generated subtitles scrolling across a news feed know that speech recognition can throw up some pretty notable errors, particularly where regional accents are involved. You can scan any amount of text and convert it to a readable computer file, as long as you don’t mind occasional foul-ups, like a history describing Hannibal as “a man of great diatinotion” at the time of the “Pfnic Wab”, or an Edgar Wallace character exclaiming in excitement, “[UNRECOVERABLE OCR ERROR]“. Machine translation can almost work in real time, for closely related languages that have substantial databases to draw on; it’s much less good if you venture further afield (as an English speaker, I found trying to converse with an Arabic speaker via machine translation was just an exercise in mutual frustration.)

Now, you may notice that a lot of these problems could be solved easily enough by the intervention of a human level intelligence. All you’d need for the speech-to-text thing, for example, is someone to listen to the speaker and, when they make a noise that the computer doesn’t recognize, insert text that matches the sounds made and the context of the speech. This is, in fact, one of the jobs that our brains do automatically for us, at an unconscious level. Similarly, the OCR problems don’t need much help beyond basic proofreading. Comprehension, spelling, and paying attention; all pretty basic human functions, things we’re taught in primary school, if we haven’t picked them up already. Basic, ordinary, everyday things – the sorts of things that, as we have previously noted, are actually damned hard to do, once you try to do them with an apparatus that isn’t a human brain.

You can do an awful lot with Large Language Models and neural networks… although neural networks have their own quirks and pitfalls; another of those cautionary tales I remember from the 1980s involved a network that had been trained, on pictures, to distinguish between NATO and Warsaw Pact tanks. It fared admirably in tests, but when a trial with fresh data began, it performed no better than chance. The network had been trained on publicity photos of NATO hardware, usually silhouetted imposingly against the skyline. Its images of Soviet vehicles, though, were mostly captured from aerial reconnaissance, looking down on them. The network had, at some time, effort, and expense, been trained to recognize the sky.⁵

AI technologies can do a lot, but one thing they can’t do – as James Fodor pointed out – is conceptualize. They can’t understand and work with the context of a question. To your average ordinary human mind, the tank problem is easily expressed – “that’s a picture of a tank, tell me if it’s an evil Commie tank or a nice star-spangled American one.” To the computer, all it has is an enormous pile of ones and zeroes which must be compared to other, even more enormous, piles, to see what patterns, if any, they have in common. It would go through pretty much the same process if it was checking to see how many different species of tree were in the picture.

People talk about the hierarchy of data, information, and knowledge, or at least they do in the low circles I move in. Your pile of ones and zeroes is data; computer systems are certainly able to organize it into information, but are they good enough to distil that information into actionable knowledge? To decide what’s the salient feature in an image that shows sixteen specimens of Quercus Robustus and one T-72 tank? That is still the province of actual human minds. Computers can be very, very good at systematizing data into information – in many instances, faster and more reliably than us poor humans – but the jump from information to knowledge requires understanding of concepts and context, which computers cannot yet do.

But, because of our human failings – specifically, anthropomorphism – we are easily conned into believing they can. Not so much by the AI promoters (though their ethics may be open to question), but by our own well-meaning but treacherous brains.

LLMs have immense amounts of data – which is one area where the ethics are questionable, since the AI industry’s attitude of “well, it’s public domain if we want to use it” has, to put it mildly, not gone unchallenged. And without a doubt, that data is analyzed, collated, and generally systematized into information. But the crucial third step – picking out the right information, in the context of the user’s question – is beyond the competence of a machine which can’t understand such concepts. We only believe it isn’t because our brains are telling us that the thing giving us natural-language responses is, somehow, an actual person. That we’re talking to a little man inside the computer. But the little man isn’t there⁶.

Even when we’re pointing out AI’s failings, anthropomorphism holds sway – increasingly, people refer to the absolute nonsense churned out by LLMs as “hallucinations”. It’s yet another helpful but misleading metaphor – suggesting, if you like, that, yeah, there may be teething troubles now, but once our beneficent techbros have persuaded the little man to lay off the funny mushrooms and the wacky baccy, everything will be just fine. Folks, the computer is not on drugs.

So, how do the AI proponents think their models are actually devising their answers? Well, I suspect that some of them genuinely believe their own hype (I swear some of these people think Tron was a documentary.) Quite frankly, it seems to me that, to a lot of these tech moguls, expertise (in technology or anything else) is not something they have, but something they buy. And, naturally, if “AI” worked, it would make that expertise cheaper for them, so they’re predisposed to believe in it. Moreover, they’re inclined to take a consumerist view of the expertise they buy, which means that, if they don’t like it, they send it back and order some that suits them better. I suspect that this is the sort of attitude that got Stockton Rush where he is today.

Some AI true believers, I think, are convinced that LLMs can lead to emergent AI – that the random rattling together of all those bits in all those processors can result in the appearance of an actual intelligence, just like the random recombinations of amino acids in the primordial ocean led to life on Earth. Who am I to say this is impossible? I would point out, though, just how long it took for those chemicals to produce life… and also that, because of all the fundamental differences between computers and humans that I mentioned earlier, an emergent AI is unlikely to regard fact-checking your article about vaccines or scuba-diving as its first priority.

I think the more realistic techbros may be hoping for a sort of “wisdom of crowds” effect – if you ask the LLM a question, the right answer must be in its data set somewhere, and surely it will stand out among all the wrong ones? Magna est veritas et praevalebit,⁷ and all that. Trouble is, the signal-to-noise ratio in the datasets available to LLMs is very, very high indeed – not least because of all the disinformation pumped into them by techbros and their political enablers. Not to mention the inherent biases in the initial data – build a model based on, say, all out-of-copyright works in the English language (which takes us to a cut-off point in the first half of the 20th century), and it will reflect all the implicit prejudices and unexamined assumptions of that corpus of literature, i.e. it will be racist, sexist and homophobic af.

The truth may be out there, but so are an awful lot of lies, and the computer has no way of telling which is which. Besides the copious disinformation being spewed across all available media, in some cases by the same billionnaire techbros who are trying to sell us LLMs, there is plenty of stuff that is just out of date, or innocently wrong in some other way, or isn’t meant to be taken seriously. We, the human beings, have things like life experience and a sense of humour that tell us, no, nutritionists don’t suggest we eat small rocks every day; no, hard-boiled eggs were not invented by Melvin Boyle-d’Egg in 1974. The computer has no life experience and no sense of humour.

And the signal-to-noise ratio is only going to get worse, because, as the dog returneth to his vomit, so LLMs are increasingly scraping their own output to expand their training datasets. That oft-quoted example where the chatbot says “there are no countries in Africa which start with the letter K”⁸ has now been so oft-quoted that it’s entrenched in the LLMs’ data hoards. To be fair, some of the AI hawkers have recognized this problem, and have responded to it by… producing more nonsense output that their LLMs have been trained to ignore, but which will be grist to the mill for competing LLMs. These people are going to usher us into a new digital utopia? Do me a favour.

“But,” I hear a voice say, possibly only in my head, “what about the generative aspect of generative AI, Steve? For you cannot deny that LLMs have produced utterances which have never been uttered before. Combinations of words that have never occurred in all of previous history! Is this not genuine creativity at work?”

So let me get back to my student days. It’s almost traditional for young men, away from home for the first time, to fall into bad company. In my case, this was the Junularo Esperantista Skota, or the Young Scottish Esperantists’ Society. (Usually abbreviated to JES, pronounced “yes”, which is also what it means in Esperanto.) Esperanto, as any fule kno, is a constructed language, devised in 1887 by L.L. Zamenhof, a Polish oculist with a visionary streak; he envisaged it as a universal second language, easy to learn (with its rational and completely regular grammar) and culturally neutral, to allow simple straightforward communication between people of all nations, and thus bring about world peace. Laudable ambitions, but doomed, because 1) Esperanto is fundamentally European in structure and vocabulary, and thus no easier than Latin or Romanian to people from outside the Indo-European language family, and 2) cultural neutrality was never going to happen, because a language is nothing without a culture, and by the time Esperanto had its own original literature and its own native speakers⁹, it certainly had a culture of its own – and what a culture it was, given that (as a fellow JESano admitted to me once) the average Esperantist is the sort of person who, if Esperanto ever did become a universal language, would immediately go off to learn Volapük¹⁰.

What has this to do with Artificial Intelligence? Well, in my second year at Edinburgh, we had to do a project (well, a number of projects, but I’m only concerned with one, here.) So I, having mastered (as I thought) phrase structure diagrams, Prolog, and Esperanto, decided to put all these things together and write a Prolog program that generated grammatical¹¹ sentences of Esperanto.

And do you know what? Maybe I had mastered that lot, because I got the darn thing to work. And, in the fullness of time, it produced an utterance of such grandeur, such majesty, such awe-inspiring grandiloquence that I myself (as you can see) was lost for words. I had to express myself in graphic art instead, producing a commemorative illustration that I have tried to reproduce herewith:-

Which, for all you speakers of manky old-fashioned languages out there, translates as “The large green aadvark sings to the brick.”

Why was I so gobsmacked by this? Well, because you could search the corpus of Esperanto literature (up to the early 1980s) without finding that sentence; nor could I imagine any way it might come up in normal conversation, not even the ones we used to have at JES. So I would bet good money (if I had any) that that sentence had never been uttered before my program output it. Genuine machine creativity!

So. Was this inspired by the muses? Did the divine spark of creativity, that which moved Milton and Shakespeare to bring forth their masterpieces, also inhere in my shonky little Prolog program that I wrote in about a month while drunk on phrase structure diagrams, backtracking, and Gaymer’s Old English?

Or is it, perhaps, just possible that there is rather more to human creativity than the mindless working-through of allowable permutations?¹²

I rather suspect there is. And I further suspect that the techbros and the LLM enthusiasts have no idea how to do it. I’m not saying it can’t be done – I don’t know that. What I am saying is that we are a lot further away from it, if it is possible, than current hype would have you believe.

I will stick out my neck a little bit here, and opine that, if it can be done, neural networks aren’t going to be the way. Neural networking is a valuable tool in actual, productive, AI work, it’s something that has produced useful results. But I suspect that trying to mimic the neuroplasticity of the human brain on a platform that is completely different in structure and operation – well, it might be possible eventually, but you’d be hard pressed to get it to do anything else useful. It’s like aeroplanes: we didn’t get them working properly until we realized the wings didn’t have to flap like a bird’s.

So. If “Generative AI” isn’t meaningfully generative, and isn’t actually intelligent (it’s artificial, though, I will grant it that) then what is it actually good for?

Not much, it turns out. LLMs are pretty good at faking things, but their output is hopelessly full of errors when you actually look at it. Because LLMs predict what’s statistically likely to be found in a given text, it’s good at faking things that follow a particular format. Trouble is, it’s easy enough to make up things that follow a format, less easy to make them make sense. I know plenty of formats, myself, and I can make up, say, a chemical formula like Kr₂Fe₃(OH)₂ or a legal citation like R v Nudgely [1959] KB 113, and they look perfectly convincing, don’t they? Unless you know anything at all about chemistry, or law.

Law is one of the first things to come up, when you think about LLM errors, because legal documents tend to have a very constrained format, so they’re easy for the model to predict. So, lawyers started to use the chatbots to generate case filings and such. And the chatbots happily produced things that looked like case filings, and any non-lawyer would agree. After all, who worries whether all those case citations are real cases, or if they actually support whatever argument you’re making? – It turns out that’s not a rhetorical question, and the answer is judges.

Similarly, cooking recipes all have a list of ingredients and a list of steps, so the chatbots can fake those, too. It takes a human being to determine if the results are even remotely edible. One example cropped up, fairly early on, of a book offering 2000 recipes safe for diabetics. I hope that no diabetics put too much trust in this one, although there was a bit of a giveaway when the (AI produced) cover described it as a “Coobook”. (AI book covers are one of the weak points, because the AI “artist” has a terrible time deciding if the letters of the title are actually letters or just random decorative elements.)

All sorts of things have predictable formats, and AI has been proposed as a replacement for humans in many of them, and it’s never a good idea. Air Canada found this out the hard way, when a customer asked about a refund policy and its chatbot gave an answer based on what certainly looked like a refund policy, but wasn’t the actual one the airline was using. Studies of AI use in computer programming have shown, consistently, that human programmers spend more time correcting the chatbots’ mistakes than they would in writing the code all by themselves.

This sort of thing, of course, has the potential to do real harm. Care plans for seniors can be chatbot generated, but of course the chatbot has no knowledge of the actual patient and their needs, and its plans may be dangerously inappropriate. The legal complications of AI-generated case filings or AI-generated policies are obvious and costly. If you wind up following an AI recipe that says, say, mix half a cup of milk, twelve cans of tuna and 500 grams of brown sugar, bake for 13 minutes at 200º C and leave to age for six weeks, your health might suffer whether you’re diabetic or not¹³. And, in this age of misinformation and disinformation, getting your news from a chatbot is not going to keep you up to date with reality – which may be a problem, if (say) your news feed tells you that the outskirts of Los Angeles are a wonderful place for a holiday right now.

There is, I suppose, one actual, valid, use case. If you want something with a predictable format, and one for which fact-checking is completely irrelevant – well, you could certainly get a chatbot to turn out any number of newspaper opinion columns, at least from some opinion-column writers. No, “Generative AI” cannot replace genuine human expertise. No, it cannot replace genuine human creativity. But I am not the only person to observe that it can, in fact, replace Boris Johnson¹⁴. Since I think Boris Johnson could be more usefully replaced by an empty space, I don’t think this is much of a win for AI.

So, we’re being sold a model for generative AI that is, at best, a Boris Johnson simulator. And it’s being sold to us by people who really believe (or really want to believe) that it is much, much more: that it is genuine expertise and genuine creativity in a can, ready to replace all those tedious and expensive humans who insist on knowing things and being able to make things – especially those tedious humans who keep insisting that the pure and brilliant vision of our billionnaire overlords needs to be constrained by actual facts. OK, maybe right now it will tell you that there are four Ns in “mayonnaise” or that eating small rocks is good for you, but these are mere teething troubles. Just needs working some details out, that’s all. Like the mechanical men of the 1930s.

There’s plenty of fallout from this situation. We’ve already seen how relying on chatbots’ output can get you sued, or hospitalized. And the implementation of this model of AI has harmful consequences outside the scope of its users, too. Graphic artists and illustrators – never exactly in the top-earning category – are having to compete with half-assed AI imagery. Independent authors (such as myself) are – well, chatbots cannot write stories, that takes things they can’t do, but that hasn’t stopped a deluge of AI-generated slush hitting publishers (see previous commentary here), and actual human beings are having a hard time keeping their heads above it. Advertising copywriting is a soul-destroying job, but it is also a job which keeps food on writers’ tables, unless they have been replaced by chatbots (who will, sooner or later, wind up costing their users a lot of money, when they make specific claims about a product or service that turn out not to be true – as Air Canada found out.) Translators are having to compete with machines, even though good translation requires an understanding of nuance, connotation, context, and shades of meaning – all of which mean nothing at all to a chatbot.

There is, too, the staggering environmental cost of these things – it takes a server farm the size of a small town, consuming enough power (and water, for cooling systems) to supply quite a large town, to tell you that Kenya starts with a K instead of a K, or draw you a picture of a woman with 86DDDD breasts and eighteen fingers. Although, as I write, the AI hype world is reeling from the revelation that the Chinese have come up with an AI engine that requires much less in the way of resources – it still produces useless crap, of course, but it’s significantly cheaper useless crap. Our AI-wielding overlords are much put out, because they reckon the Chinese system has scraped up some of their intellectual property to do its stuff. That scuffling noise you hear in the background? It’s the sound of all the writers whose work was used without permission for AI training, all trying to get the world’s smallest violin.

AI is something lots of people would like to believe in – including me; let’s face it, I’ve read many of the same SF novels as Elon Musk. But it’s not here yet, and we’re a long way from it. But it is all too easy to believe that it is here. Not so much because of all the hype – aggressive though it is; I’m old and cynical enough to believe that you can’t trust the likes of Musk or Sam Altman as far as you could spit a rat into a headwind. The problem is our treacherous but well-meaning brains, conning us into seeing humanity and consciousness where there is none. Your lawyer drafting AI filings isn’t thinking “Oh, nobody reads this stuff anyway, I can send in any old rubbish.” No, that lawyer is thinking that the chatbot who sounds all so human… will understand what he needs, will test its arguments and check its references before coming out with its answers. That lawyer is relying on the common sense, the wisdom, the understanding of the little man inside the computer.

The little man who isn’t there.

1. The difference is that an interpreted language like BASIC generates the binary code one line at a time, whereas a compiled language like, say, Pascal reads the whole program at once and generates a complete version of the binary code from that. Pascal and BASIC are both very old languages indeed, and computing has rather Moved On from here. The main advantage of an interpreted language is that you can watch it go wrong in real time; a compiled language spits a “COMPILATION ERROR” back at you when it’s finished, and you have to dig through the program all over again to find what it is you’ve screwed up. Modern development tools will at least highlight their best guess as to the source of the error, although the process of fixing it often devolves into a lengthy and soul-destroying game of Spot The Typo. Since modern computing involves a lot of object-oriented design, not to mention linking to precompiled software libraries that may or may not even be on the same machine as the main program, it’s safe to say that interpreted languages have rather fallen by the wayside. Progress. ↩︎
2. I’ve seen a figure of 1.6 zettabytes quoted recently. I do think that qualifies as rather a lot. ↩︎
3. Well, why should the Jews get all the space lasers? Eh? ↩︎
4. Where I once saw a message, hastily scribbled on a chalkboard, saying “The Fuzzy Reasoning Group may meet hereabouts sometime.” No, it’s not actually relevant, I just thought it was funny. ↩︎
5. This is a neglected feature of the debate about creating autonomous war machines – armed robots that make their own decisions about when to use their weapons. Some people argue (quite rightly) that this is a fundamentally appalling idea and should never be done. Some people argue (also quite rightly) that, given the inherent bias in so many data sets, you’re likely to have your AI killbot deciding someone is a legitimate target just because of their facial hair or the colour of their skin. I am going to argue (also quite rightly) that, given the way neural networks learn, you’re likely to have your AI killbot deciding to murder someone because it’s 15:01 and they’re standing 12.3 metres from a man in a purple shirt, because you just don’t know what the network is going to find important and actionable. ↩︎
6. We shall gloss over those cases where it turned out the little man was actually there, and he was one of a slew of underpaid call centre workers in some offshore digital sweatshop ↩︎
7. “The truth is mighty and shall prevail”. Don’t let anyone tell you I’m not merciful to non-Latinists. ↩︎
8. Sometimes adding “The closest is Kenya, which sounds like it begins with a K but actually begins with a K.” Do you seriously expect me to believe that a sentence like that was uttered by someone or something who understands what words mean? ↩︎
9. Boy Esperantist meets girl Esperantist at an international convention, love blossoms among the dudek reguloj, and in the fulness of time, a new generation is born. And if one parent is (say) Norwegian, and the other is (say) Korean, what language do you think they have in common? And what language will they use around the home? And what language do you think their kids end up speaking? One of my fellow JESanoj was a native Esperanto speaker. Nice bloke. ↩︎
10. At the time, the second most popular conlang. Esperanto had an estimated 500,000 speakers then; Volapük had 7. ↩︎
11. I had to hand code my way around a dozen exception cases, even with the very limited subset of the language I was working with. Completely regular grammar my arse. ↩︎
12. I’m reminded of an episode of QI in which Stephen Fry announced, grandiosely, that he was going to do something which had never before been done in the history of the universe – and then, with great flourish and fanfaronade… shuffled a pack of cards. The panelists were notably unimpressed. Fry explained that (what with factorial 52 being an extremely large number and all that) it was effectively certain that no pack of cards had ever been in that particular order before, nor would one ever be again. “So what?” was the attitude of the panel. And mine, too. ↩︎
13. There is a weird side case, and I don’t know if it’s done with LLMs or not, but people have been using AI for identifying edible fungi. And while it can correctly identify that fungi are edible, it has a rather less good track record when it tries identifying fungi that – as Sir Terry Pratchett might say – are only edible once. It makes mistakes, and not trivial ones; it makes the kind of mistake where, as novelist and nature expert Ursula Vernon observed on Bluesky, “they will be able to pour your liver out of your corpse at the autopsy”. ↩︎
14. It would not surprise me in the least if Boris Johnson was already using ChatGPT or similar to produce his columns. After all, actually writing all that stuff would seriously cut into his drinking and shagging time, wouldn’t it? ↩︎