Could AI engineer a pandemic?

Articles published here are typically written by individual Torchbearer Community members, with input and editing from others. Views expressed do not represent an official position of Torchbearer Community as a whole unless otherwise noted.

The recent Covid-19 pandemic was terrible, taking over 7 million lives, but the majority of us survived. Humanity has, fortunately, survived all previous pandemics such as the bubonic plague and Spanish flu. We have again and again demonstrated our ability to overcome the limitations of, and threats from, nature. However, we may not survive a future pandemic if it is intelligently engineered with the help of advanced AI to cause maximum destruction and death.

There is ongoing research where dangerous pathogens, like viruses involved in previous pandemics, are modified in ways that enhance their functions. Fortunately, engineering pandemics is difficult, requiring a lot of resources and expertise. While expert biologists today may be able to engineer extremely dangerous pathogens, such pathogens have not yet caused all of us to die.

But things are changing. The cost of the required materials (like DNA segments) is declining while biology equipment is becoming more accessible. Sufficiently capable AI systems will lower the barriers further, providing the necessary expertise in place of human experts. Material and know-how barriers to engineered pandemics are disappearing.

The AI developers themselves are worried about this. Their safety frameworks, such as OpenAI’s Preparedness Framework and Anthropic’s Responsible Scaling Policy, discuss how their AI models may be misused to acquire bioweapons. Anthropic further explains in a blog post why they take this so seriously.

In fact, models are already displaying concerning levels of biological capability. Anthropic’s AI released in February this year, Opus 4.6, (see the system card) demonstrated ability to assist biology PhDs in “new, creative types of biology” or help them determine the required protocol for reconstructing a challenging virus. Yet Anthropic doesn’t consider current capabilities concerning enough to require stronger security measures. AIs released after Opus 4.6 possess even more dangerous dual-use capabilities. If such capabilities continue to advance, even people with only basic understanding of biology will be able to develop novel pathogens.

Image: A woman wearing a protective mask walks past a coronavirus-themed mural in Solo, Central Java province. Antara Foto/Mohammad Ayudha

In order to avoid catastrophe, terrorists and other untrusted actors should absolutely not have unmonitored access to AI models that could help them engineer pandemics. Anthropic and other frontier AI companies have a solution for this: they simply do not let people run their AIs on their own computers. AI users, such as those using chatbots, interact with AIs that are run on servers owned by the companies, which allows for oversight and control over how these products are used. But people have found many ways to circumvent existing safeguards through jailbreaking, tricking the AIs and bypassing their ethical guidelines, allowing them to provide dangerous information and instructions.

Other AI developers, however, simply release their models openly, without any restrictions or control. When a highly capable AI is released, outperforming all previous models, it takes just 3 months on average until a completely open model is released with matching capabilities. If Anthropic announces that its AIs can be used to help engineer novel bioweapons, it might take just a few months until terrorists can get unfettered access to models with similar capabilities!

Unless, of course, there are proper regulations in place that prevent such unrestricted AI access. These regulations do not yet exist, neither in the US or China where frontier development is currently happening, nor internationally (although some approaches are being debated in the UK and Canada).

You may think that no one would be stupid or malicious enough to engineer a pandemic that could end civilization. If that is the case, I have bad news. First, historically, there have been groups willing to do this. In July 1993, on the roof of an eight-story building in Tokyo, Japan, a doomsday cult called Aum Shinrikyo attempted to start a pandemic.

Fortunately, they made critical errors and ultimately failed but could arguably have succeeded in killing thousands if they were better prepared. Since they had the necessary resources and were not attempting to release novel pathogens (which requires advanced research), they might have succeeded if they had unrestricted access to current frontier AIs.

Additionally, various terrorist groups have shown their willingness to commit atrocities. Rogue nation-states may be tempted to use engineered pandemics to hold our entire civilization hostage. Some bad actors want to destroy everything. Others would threaten to do so if pressured.

Secondly, civilization-ending pandemics could be released by accident. Capable AIs would enable more dangerous research on novel pathogens. If novices become able to do such research, they may not employ proper security, and release dangerous pathogens by accident.

Engineered pandemics are already a serious threat, even without involving AIs. Current AIs are already able to assist human experts in biological research on bioweapons, but do not appear good enough yet to support non-experts. AI capabilities are advancing extremely fast, and some developers are giving absolutely everyone complete access to their models which are just 3 months behind the frontier on average. The situation is grim.

We at Torchbearer Community are very concerned about the existential threat of superintelligence, which could outsmart us all. Yet, AIs could enable humans to end civilization long before the arrival of superintelligence. We need to get our act together now. In our next article on this topic, we’ll explore actions that can and should be done. Stay tuned.