AI, the Church, and the Bomb

This week a landmark conference occurred at the Vatican. For three days a gathering of 200 Nobel laureates, leaders in AI, experts on nuclear war, former heads of state, and leaders of the Catholic Church discussed the growing influence of artificial intelligence over the use of modern weapons. The conference culminated with the writing of the Rome Declaration, a call for “urgent, sustained and good-faith negotiations leading, within an agreed and time-bound framework, to the verifiable and irreversible elimination of nuclear weapons.”

The conference began as a gleam in the eye of Dr. James Muller and inspired by Pope Leo XIV’s new encyclical, “Magnifica Humanitas,” dedicated to the care of humans in the age of artificial intelligence, and the Pope’s vision of an “unarmed and disarming peace.” Muller is a member of what the Boston Globe called the “Boston Brain Trust” of antinuclear physician activists (full disclosure: as a member of that group I played a marginal role in planning the conference). Muller, a co-founder of the International Physicians for the Prevention of Nuclear War which had been awarded the Nobel Peace Prize in 1985, reached out to the usual suspects in the nuclear arms race—the U.S. and Russia. He also spoke with representatives of the United Nations and the world’s leading moral authority against nuclear weapons, the Catholic Church. As Muller noted, “The US and Russia hold 90% of the world’s nuclear weapons, with no arms control treaties currently in place. Renewed dialogue between them is essential to eliminating this existential threat to life.” Another member of the “Brain Trust,” Dr. Ira Helfand, put together the goal of the conference with surgical precision. “Don’t give AI the bomb.”

How may a nuclear war start? For seventy years it had been planned (and possibly prevented) by the superpowers’ policy of Mutually Assured Destruction (acronymized as MAD). If one made a nuclear assault, the other would respond, and by the end of the day the US and the USSR would have destroyed the world. But time changed that. Today nine countries have nuclear weapons at the ready. Predicting who, what, when, and why a nuclear war might start is virtually impossible. Increasing numbers of nuclear nations incentivize their neighbors to become nuclear wannabes who in turn are induced to “modernize” their armaments. Treaties which reduced world weapons by 75% have now collapsed. More weapons mean a greater chance of use. Subnational ideologues, armies, and terrorists given half a chance would be more than happy to use them. And then there is the problem of human fallibility. How long will it be before an accidental detonation unleashes unspeakable harm? No control system is perfect. The probability of failure increases with time. The Doomsday Clock, an analog estimate of the probability of nuclear war set each year by the Bulletin of Atomic Scientists is now set at 85 seconds. Consider the nuclear saber rattling of Donald Trump and Vladimir Putin this year and you know why. Enter the meteoric rise of AI to raise the ante on the bet of our survival.

AI has captured the imagination of war planners. Warfare is becoming automated. A scholar writes in the Army War College journal War Room that we are at an Oppenheimer Moment, a time like that of the first atomic explosion from which the world could not turn back. AI companies are in a race with one another for profits. Governments compete with each other for political and economic dominance. Machines in the meanwhile are rapidly outstripping what humans can do.

Graph credit: https://aistats.live/. “You and me” line added by author. Fair use.

As AI drive weapons in war, decisions will need to be made in decreasing windows of time. Humans will no longer be necessary, since AI systems are approaching the condition of being able to think for themselves. As Yoshua Bengio, the co-chair of the United Nations’ Scientific Advisory Panel on AI, and one of AI’s godfathers, put it, “We have built machines that are seeking goals.” Their own goals, not ours. AI systems are becoming vastly more intelligent than any human being, and capable of manipulating us, cheating, lying, all the while being able to make mistakes. They can program themselves and cooperate with other AI systems. And they can do all this without a trace of moral concern about how they affect humans.

Yet AI regulation at the moment is a pipedream. Common sense says that AI companies should be required to show that their products are safe before deploying them. Bengio says, “We do it for planes. We do it for meat, for cars, for drugs.” He compared our situation to the problem of driving a mountain road in darkness. The reasonable tactic is not to speed up or stop, but slow down enough to see what’s ahead. “We need guardrails to make sure we drive that development in a direction that’s actually beneficial for people, for humans, for society, and for our well-being.”

And what if Bengio’s dark mountain road is analogized to the world turning over the command and control of its nuclear arsenals to a network of superintelligent machines? That was the terrifying topic facing the Global Nobel Laureates Assembly at the Vatican this week. Given the existential threat of machine driven nuclear war, it was perhaps not surprising that this bouillabaisse of Nobelists, international experts, former heads of state, and religious leaders could come together and write the Declaration of Rome.

They called for action to head off “the next arms race,” both AI and nuclear. AI development needed to be responsible, accountable, transparent, and aligned with humanity’s interests. Under no circumstance should any system be permitted to self-improve without monitoring and the means to shut it down. An international treaty needed be adopted to prohibit integrating any AI system into systems of nuclear command, control, and launching. Governments and leaders needed to step up, and people need at every level to be educated regarding the promise and peril of AI. Some argued that AI could not, and should not, be halted. But there was unanimity that by removing nuclear weapons themselves from the equation (“the hardware,” as James Muller put it), nuclear war itself could be prevented. As the Cardinal Vicar of Rome, Baldassare Reina, said at the closing, “Nessuna macchina, nessun algoritmo, nessun sistema autonomo può essere posto al centro di decisioni dalle quali dipende la sopravvivenza dell’umanità.” (“No machine, no algorithm, no autonomous system can be placed at the center of decisions on which the survival of humanity depends.”)