Waving or Drowning

I had mixed responses to Mustafa Suleyman’s book “The Coming Wave”.

I’ll start with a positive: the central argument is stated with admirable lucidity.

According to Suleyman, technological advances come in waves. The last wave brought us ubiquitous computing, the internet, and social media. The coming wave is bringing us AI and synthetic biology, together with advanced robotics, quantum computing, and new sources of energy. This wave is rapidly gaining momentum.

The power of these emerging technologies and the speed of their likely diffusion present us with a series of dilemmas.

If we eschew these technologies – admittedly an unlikely scenario – then we will severely damage our prospects of tackling the world’s most pressing problems, including climate change.

On the other hand, embracing these technologies also carries risks. In the wrong hands, they could lead to catastrophe. Yet the fear of catastrophe could lead to an opposite, equally bad outcome: the emergence of tech-enabled tyranny. The only safe way forward is what Suleyman calls “containment”: a combination of regulation, cultural change in the tech industries, popular pressure, and international collaboration, all working together to ensure that the coming wave works for our shared benefit and that we escape the worst risks. Not so much “move fast and break things” as “move carefully and try to avoid too much collateral damage”.

Suleyman co-founded the AI company Deep Mind (subsequently acquired by Google) and is now CEO of Microsoft AI; but earlier in his career he worked for Ken Livingstone, when Livingstone was Mayor of London. This combination of experience is unusual. He is well placed to discuss both emerging technology and its impact on politics and government.

The book gets off to a slow start, but hits its stride in Part 2, which outlines the main components of the coming wave, with AI and synthetic biology at its centre.

Those who discuss AI risk tend to fall into two, often polarised camps. Some focus on harms that are already apparent, e.g. biased training data leading to biased outcomes. Others turn their attention to more long term or speculative risks, stemming from the possible development of artificial general intelligence (AGI), i.e. AI that matches or surpasses human performance across every domain. Suleyman takes a middle course. He proposes a “modern Turing test”: could an AI successfully act on the instruction, “take $100,000 and make $1,000,000 on Amazon in just a few months”? An AI that could pass this test would have demonstrated what Suleyman calls Artificial Capable Intelligence (ACI). Suleyman evidently sees ACI as a realistic near-term prospect. He thinks we should spend more time thinking about its implications and less debating the possible emergence of AGI or speculating about machine consciousness.

The main recent developments in AI are widely known. Suleyman’s discussion of synthetic biology may be less familiar. He outlines three developments: human genome sequencing (the cost of which fell from $1 billion in 2003 to well under $1,000 by 2022); the editing of DNA sequences using CRISPR; and DNA synthesis, involving the creation of novel forms of DNA by way of DNA printing. The potential for widespread dissemination of this latter technology is the most alarming element of Suleyman’s account.

The other technologies he discusses (including quantum computing, robotics and nuclear fusion) are passed over more quickly, though Suleyman refers to the potential for quantum computing to break the encryption systems on which modern online finance depends. An important message throughout is that we should not think of these various technologies in isolation, since much of their impact depends on their mutual interaction.

The potential social and political impact of all this is unpredictable. In economic terms, Suleyman expects to see the principal share of the rewards going to a small number of tech companies. Some of these may come to play a broader, quasi-governmental role; here, Suleyman refers to the history of the East India Company. Politically speaking, he considers that these technologies will increase the power both of states with an appetite for surveillance, and of non-state actors promoting fragmentation. Suleyman coins the word “Hezbollahization” for a possible future in which the latter tendencies win out. In case we are perplexed by the argument here – which seems to be identifying a dynamic that pulls in opposite directions, being both centrifugal and centripetal – Suleyman reminds us that social media created a few corporate giants, and a million tribes.

In Suleyman’s view, some of the worst catastrophic risks are those associated with cults or terrorist groups. He invokes the 1990s Japanese doomsday cult, Aum Shinrikyo, which released the nerve agent sarin on the Tokyo subway. Suleyman fears that, armed with a combination of ACI and DNA printers, a future such group could engineer a deadly pandemic – “gain of function” research of the most nightmarish kind.

Suleyman’s ten-point plan for containment – outlined in the final part of the book – all sounds entirely reasonable. The difficulty is that there is currently fierce geopolitical competition for dominance in these emerging technologies. Put simplistically, Suleyman’s proposals would require the US, China, and India, to become more like the EU. If anything, the current trend is in the opposite direction. To be fair, Suleyman recognises these difficulties, advocating Gramsci’s prescription of pessimism of the intellect combined with optimism of the will.

The book has an underlying political vision which I find persuasive. Modern Western democracies face two opposing dangers. One is that they tilt towards authoritarianism and become increasingly like China. The other is that they are weakened by loss of legitimacy, widespread disengagement, inability to take effective action, and social fragmentation, becoming increasingly like Lebanon. Political virtue consists of finding the Aristotelian mean that avoids these extremes.

I can’t say that it’s exactly an enjoyable read. The writing is functional rather than pleasurable, despite the assistance of Suleyman’s co-author Michael Bhaskar. There are passages that don’t say very much more than “everything is changing very fast, and the consequences are uncertain”. At times, it’s like reading a slide deck that has been turned into extended prose. On occasion, the style reminded me of Corporate Memphis: all big shapes and bold colours, but not enough light and shade, leading to a sense that everything has somehow been flattened.

More fundamentally, Suleyman’s picture of what we can expect if everything turns out as well as can reasonably be expected is still a daunting one. Consider this:

We are going to live in an epoch where the majority of our daily interactions are not with other people but with AIs … They will be our personal intelligences, our companions and helpers, confidants and colleagues, chiefs of staff, assistants, and translators …Culture, politics, the economy; friendship, play, love: all will evolve in tandem

Or this:

The human genome will be an elastic thing, and so, necessarily, will be the very idea of the human itself.

Passages like this put me in mind of CS Lewis’s book “The Abolition of Man”, especially the two fundamental questions that Lewis raises in its final chapter: first, whether what is heralded as the mastery of humans over nature really equates to the mastery of some humans over others; and secondly, what moral vision (if any) will guide those who seek to design the humans of the future.

I’ve no doubt that emerging technology will be highly disruptive. In this Substack, I’m interested in any resources that can help us make sense of what’s coming. That means learning from thoughtful industry insiders like Suleyman. But it also means engaging with with a much wider array of voices: we need to hear from poets and philosophers, novelists and lawyers, filmmakers and musicians, and many more besides.