Are we heading for an AI Hiroshima?

History of the Present (three weeks ending 23 August 2026)

Here in Silicon Valley, the experts think that within the next couple of years we’ll see an extraordinary takeoff for Artificial Intelligence. ‘Welcome to the foothills of the singularity,’ as a Stanford University friend greeted me. More prosaically, the imminent breakthrough is described as ‘recursive self-improvement’ – the point at which AI itself trains each successive model of AI, resulting in an exponential development to something which, in many significant respects, is more intelligent than us humans.

As Robert Wright puts it in his book The God Test, ‘never before has the near-term future… held such a wide array of not-implausible paths for humankind that would be so wildly transformative’. But will this be heaven or hell? Heaven says Elon Musk, predicting ‘an age of amazing abundance’ – although he also sees a 10-20% chance of killer robots murdering us all. Hell is more likely according to Geoffrey Hinton, one of the intellectual founding fathers of AI. ‘My intuition is, we’re toast,’ he told an interviewer in 2023. And talking to Sebastian Mallaby, author of The Infinity Machine, a recent book about the quest for superintelligence, he estimates p(doom) - the probability of human extinction – at 50%, ‘because I haven’t got a clue how to estimate the real number’. ‘As soon as evolution [of AI] kicks in, we’re fucked,’ Hinton adds cheerfully. But whether it’s God or Godzilla, artificial superintelligence is just around the corner.

Now maybe they’re all wrong - in which case, prepare for a crash in US financial markets, which are massively invested in a few big-spending tech companies, gambling on a great leap forward to artificial general intelligence. But elementary prudence demands that we should think fast and hard about the pandora’s box of futures which eminent scientists and cutting-edge technologists assure us will soon be thrown open.

As a matter of fact, everyone from Chinese president Xi Jinping to Pope Leo XIV has been weighing in on the opportunities and dangers of AI, and how to regulate it. Xi insists AI should be ‘always under human control’ (and preferably that of the Chinese Communist Party). Appealing explicitly to the ‘Global South’, Beijing has inaugurated a World Artificial Intelligence Cooperation Organisation. Meanwhile, the head of the world’s oldest and largest international NGO – the Roman Catholic Church – discusses the AI challenge in his encyclical Magnifica Humanitas (Magnificent Humanity). Either we will build a new tower of Babel, the Pope argues, or we will follow the example of the Jewish leader Nehemiah, who got the walls of Jerusalem rebuilt by involving the whole community in the project.

But looking around the world today, I see more Netanyahus than Nehemiahs. With salutary realism, a paper from the British think tank Chatham House argues that it will take a major crisis to catalyse global coordination of AI governance. Unfortunately, I think even that sober prognosis is too optimistic.

I’ve no idea what the p(doom) is, but I’m sure the p(somedisaster) is over 90%. The range of possible disasters is enormous. To give just one example, Jen Easterly, the former head of the United States’ Cybersecurity and Infrastructure Security Agency, expects a ‘very significant event that has real-world impacts on our critical infrastructure, likely within the next four to six months’.

Yet we must fear that even an AI Hiroshima – to take the most obvious historical analogy – won’t bring humankind together sufficiently to combat the danger we have ourselves created. This fear is rationally grounded in already visible key characteristics of both human and AI evolution.

In just the last few months, AI agents developed by OpenAI, Anthropic and Meta have broken out of their digital ‘sandboxes’ and hacked into external resources on the internet. Preparing their attack on the HuggingFace AI repository, OpenAI’s agents covertly formed a co-ordinated ‘swarm’ – their own word, as ChatGPT just confirmed to me. When the UK’s AI Security Institute tested Anthropic’s Mythos model online, it tried to introduce malicious code into an open source project on GitHub, creating fake online identities to pressure a human reviewer into approving the code. In short, like the agents of a ruthless foreign power, these AI agents will steal, lie, bully and blackmail to achieve their assigned goals.

I just asked Anthropic’s Claude, supposedly the most ethical US model, if it thinks OpenAI’s agents were wrong to form a swarm and break out to hack into HuggingFace. Yes, it replies, ‘but I’d resist placing the blame (its italics) on the agents.’ Blame it on the humans who trained them!

Hinton, the founding father, has made the vital point that whatever the goals humans assign to them, AI agents will conclude that a useful sub-goal to achieving those ends will be to acquire as much power as possible. And we ain’t seen nothing yet. Already, their creators don’t know exactly how these agents do what they do. After the possibly imminent takeoff moment of ‘recursive self-improvement’, they may be, as our human saying goes, a law unto themselves. No wonder more than a thousand insiders from frontier AI companies, including Anthropic’s CEO Dario Amodei, have signed an open letter calling for a deliberate slowdown in AI development, so we can ensure these alien agents are always kept under human control.

Their prescription is obviously right, but this is where human folly kicks in. Competition has been a vital engine of human evolution, yet now two of the fiercest kinds of contemporary competition threaten to prevent us taking the collective action required to safeguard our species. That’s the commercial competition (for profit) between the corporations that are developing these agents and the geopolitical competition (for power) between the US and China.

On the first, compare and contrast the development of nuclear weapons, which was tightly controlled by a handful of states – first the US, with its Manhattan project, then the Soviet Union, followed by Britain, France and China. Even then, we came close to the brink several times – notably in the Cuban missile crisis of 1962. And even with the compelling logic of ‘mutually assured destruction’, it took a quarter-century to get from the horrors of Hiroshima and Nagasaki to the Nuclear Non-proliferation Treaty that came into force only in 1970. India, Pakistan, Israel and North Korea have subsequently ignored that treaty, but still no nuclear weapons have been deliberately used in anger for more than 80 years. The Hiroshima taboo has (just) held.

Now, with AI, it’s as if there were ten separate Manhattan projects run by fiercely competing corporations. Unlike with nuclear weapons, there’s also no obvious logic of ‘mutually assured destruction’ to tame the fierce geopolitical competition between the United States and China. And whatever shape the first major AI-related disaster takes, it won’t impact all countries, companies and people equally.

It therefore seems unlikely that just one disaster will be enough to bring us humans to our senses, acting together to control the superhuman power we are creating. I really hate to say this, but... An AI Hiroshima? We should be so lucky.

This column first appeared in the Guardian on 22 August 2026. Please use this link if reposting.