I’ve heard the argument a few times recently that people don’t want to believe in x-risk for emotional reasons. Someone gave a talk making this claim at an EAGx conference, and I read this in 3quarksdaily’s review of Yudkowsky and Soares’ new book the other day:
“However, the reason people reject the argument [in If Anyone Builds It, Everyone Dies] is not on account of the strength of its logic. It is all emotional. People in general do not like to think about extinction, or even risk in general. They would rather posit all kinds of emotional counterarguments.”
The speaker at the EAGx conference argued that, because of this, those of us who are significantly concerned about the loss of all life from rogue AI should instead talk more about mundane harms that don’t trigger emotional reactions, such as inequality, job loss and bias.
Regardless of whether the alternative strategy is prudent, I’d like to address the original claim here. I fully accept the idea that “it’s all emotional” when it comes to accepting or rejecting arguments. Emotions are the gatekeepers that only allow ideas that feel right to become part of our worldview, especially when dealing with such visceral issues as death and extinction. But I think we underestimate the ways in which emotions drive us towards as well as away from these topics.
The idea that people recoil from existential risk because it’s unpleasant to think about neglects humanity’s long-standing embrace of doom!
Let’s give a few examples of where we lean into our dark side:
Big chunks of the climate movement really love the apocalyptic framing. They call their protest movements “Extinction Rebellion”; “The World Is Burning” has become official UN messaging; and activists have staged mock funerals for humanity, carrying coffins labelled “Our Future”.
As much as climate change is genuinely an existential risk for some species, and may feel like an oncoming “Great Flood” apocalypse for some Pacific Islands, it’s unlikely to pose anything close to an existential risk to humans (according to our best models). But this doesn’t stop the activists. The people who care most about climate risk frame it in existential terms, and there’s at least some evidence that this might work!
Climate doom inspires the emotions that seem to be driving a genuinely global movement, even if they sometimes overshoot the science.
Millenarian or “end-of-the-world” religions, with belief in some kind of apocalypse and redemption for the chosen few, are big. This is despite famously getting you to do crazy stuff like donate all your money to fund the lavish lifestyle of the leaders, cut off relationships with your family, and/or leave your fields untilled in order to rise up in armed opposition to a vastly more powerful and organised state.
A really extreme doomsday cult might only attract a few particularly susceptible individuals, and Heaven’s Gate and Aum Shinrikyō fortunately weren’t popular or meme-able enough to reach a broader public, but religious movements with slightly apocalyptic undertones seem pretty brilliant at drawing people in. Narratives of doomsdays and messiahs have always thrived, from Ragnarok to Revelations in the ancient world, through anti-Imperial sects in China and Japan, to Islamic death cults.
This is still happening. Pentecostal movements across Africa, where regular failed rapture predictions cause unexpected waves of social panic, are growing fast and embracing social media. In the fastest growing major faith, belief in the Day of Judgment (Yawm al-Qiyāmah), where mountains level, stars fall from the sky and the heavens roll up, is a universally held belief.
Not only does preaching about the apocalypse not put people off, it inspires and draws new believers into new faiths!
Finally, anyone who suggests that people dislike thinking about doom should take a look at some of the most popular films, novels and games from the past few decades.
Who can deny the childhood excitement at the primordial fear of pure doom promised by Dark Lords like Sauron, Darth Vader, or Voldemort in popular fantasy?
Even when we grow up and “put away childish things”, many humans start enjoying films about nuclear destruction (Dr. Strangelove, Chernobyl), asteroids (Deep Impact, Don’t Look Up), or aliens (Arrival).
And, to those who might suggest that we’re just into the jeopardy of doomsday scenarios, we even get excited by post-collapse scenarios! Who hasn’t fantasised a little about living in the post-apocalyptical hell-scape of books like The Road, films/TV like the Matrix, Children of Men, or Mad Max, or games like Fallout, The Last of Us, or Death Stranding?
Does this sound like the media consumption of a species that hates thinking about doom?
In the absence of higher-quality direct evidence about whether people actually enjoy these ideas, I ran a quick survey in a rationalist discord group (which, I can assure you, is the methodological gold standard) and asked whether people agreed with the statement: “I find it very unpleasant to think about “doom” or “the apocalypse””. Over half of respondents replied with the “strongly disagree” option.
A few quotes from the responses:
“Hello darkness my old friend”
“Idealized apocalyptic scenarios where I’ll be the first to get bitten by a walker are nice”
“I think about the Zombie apocalypse more often than I think about the Roman Empire.”
So it’s not just escapism in fiction, many people actually find thinking or fantasising about the apocalypse pleasant! The people who find it unpleasant gave reasons like:
“I sometimes get images of the type of Apocalypse where nuclear warheads are randomly flying through the sky and I evaporate in an explosion. That’s the one that I don’t like. A lot in my future that I’m excited about and it would be a huge bummer to not get to experience it.”
“I thought it was fun until I started to believe it was real”
“I’m not prepared for the timeline where I’m eating my neighbor to survive”
My interpretation: more nebulous and stylised “doom” ideas are actually not that unpleasant to think about, but when it gets too real, or when we think very carefully about the apocalyptic logistics, the fantasy can slip a little.
Of course, this is mostly young people with (I presume) high openness to experience, and you’re going to get different results if you survey 62-year-old civil servants looking forwards to their retirements. But this still suggests that humanity’s doom fascination is still going strong.
Time for a spot of armchair psychology/anthropology. Firstly, a few intuitive reasons why humans might be particularly inclined towards doom narratives:
They are meme-efficient. “Everyone dies” is far easier to communicate than “we experience gradual empowerment over two centuries with 28% probability.”
They’re impersonal. Global catastrophe doesn’t threaten the ego or throw you into a melancholia in the same way that a cancer diagnosis might.
They inspire meaning rather than powerlessness. Many people really like the idea of an existential struggle. Who wouldn't prefer to fight for our planet’s survival over meeting monthly performance targets or saving enough money for a managed decline in an old people’s home!
There are also a couple of academic theories that might support my observations here. Like a lot of psychology, our attraction to doom is probably best explained through a mix of something boring and testable, and something cool and not testable. The cool one is Terror Management Theory—a sprawling post-Freudian idea that all of our actions can be explained by the fear of, and desire to overcome our own mortality, where we craft elaborate narratives and rituals to navigate these conflicting emotions. Despite being mostly unfalsifiable, it does capture some of my emotions around death. I’m a simple man: all I want is a sublimation of mortophobia into an epic mix of art, heroism, and ideology that makes the inevitable death of the self fade in importance.
The boring testable explanation comes from media analysis, where messaging is analysed across the axes of valence and arousal. Valence can go both ways—despite the cliche of “doomscrolling”, positive news does attract a lot of attention online. But the ‘arousal’ dimension is more predictive of a message “going viral” and provides a useful explanation for doom’s persistent appeal. If a message triggers high physiological activation through fear, kittens, anger, awe, boobs, or doom, it’s way more likely to attract attention, stick in your mind, and get shared.
I’m often surprised by the number of people who have a very strong inclination to think about x-risk. In fact, I’ve recently had conversations with people who had barely heard the arguments for AI x-risk and were already persuaded!
I'm fairly new to this AI messaging game, but I’d confidently make the case that we shouldn’t completely avoid dark or doom-laden messaging, at least for some audiences. I’d even suspect that using existing attraction to doom might be a neglected approach—targeting the doom-curious with fact-based messaging.
When your argument rests on the eminently reasonable, but also very “sci-fi apocalypse” idea that misaligned AI has a significant chance of destroying humanity, you have an important strength! Sure, you could make economic arguments about job losses, about privacy and control, or about AI having the values of your interlocuter’s hated political enemies. But this can feel a bit like Hafþór Björnsson auditioning for deep character roles—it doesn’t play to your strengths. You have a pre-made high-arousal, epic, yet depersonalised narrative, so why not use it!
In this respect, I think “If Anyone Builds It, Everyone Dies” has done its job pretty well. Despite the dark and serious subject matter, the argument through stories and analogies does allow the book to capture something of the scale and feel of the epic doom narrative, and make reading it actually kinda fun, especially for the darkly intellectually curious.
People still disagree with the book for many reasons, but I wouldn’t generally put these disagreements to an unwillingness to think about the dark consequences. I’d probably blame things like a general dislike of the Yudkowsky aesthetic, a strong denial that ASI is even possible, or a reluctance to accept arguments that go against the rest of your worldview (e.g. technology tends not to kill everyone, most doom predictions are wrong).
But I think that AI Safety messaging can build on humanity’s darker, more doom-loving side. We can sell inspiring, high-stakes messages with a hint of apocalypse to the young, action-oriented and open-minded, while selling “serious” economic messages to the stability-seeking 50-somethings whose interpretation of “existential risk” is a 15% drop in their savings portfolio. This might look like subtle narrative shifts, but it could look like “embracing the dark arts”, learning from how armies sell narratives to prospective soldiers, how radical religious movements attract new followers, and how extreme political movements sell fear-based narratives to get into power.
Do you have the courage to develop effective, high-arousal messages that combine our doom proclivities with a practical call for action? I hope so, my friends, because darkness is falling, and the fate of our universe may depend upon it.
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.