Why AI Leaders Keep Saying Their Product Will Destroy Humanity

There is a rhetorical genre that has become so common in AI discourse that it barely registers as strange anymore, even though it is one of the strangest things any industry has ever said about itself. A leader of a frontier AI company states, calmly and on the record, that there is a meaningful probability his own product ends civilization. Once, Anthropic’s CEO is said to have told an interviewer that he puts the chance of civilizational collapse from AI at 10 to 25 percent, in the same conversation in which he named The Making of the Atomic Bomb as his favorite book. Versions of this statement have come from nearly every major lab leader over the past three years, with probabilities ranging from a few percent to half.

The natural reaction, and I see it daily from thoughtful people, is bafflement. Why would anyone build something they believe has a one in four chance of ending humanity? The question is usually asked rhetorically, as if no answer could exist. But it has an answer. Several, actually, and they fit together. The bafflement comes from reading these statements as risk assessments. They function as something else entirely, it’s about boosting the company by boosting their model’s abilities.

Start with the simplest observation. Nobody assigns an extinction probability to a toaster. Stating that your product might end civilization is, structurally, the most extreme capability claim it is possible to make. It asserts that what you are building is the most powerful artifact in human history, which is to say the most valuable. And unlike a revenue projection, it can never be audited, never be marked to market, and never trigger liability, because it is a prediction about the far edge of possibility rather than a statement of present fact. A claim that simultaneously maximizes implied value and minimizes accountability is not a confession. It is the safest possible pitch.

This is why the statements cluster among people raising capital and recruiting talent in a competitive market. The danger is the differentiation. A lab whose product might merely be useful is competing on price. A lab whose product might be the last invention is competing for the future itself.

Here is how a sincere person holds a 25 percent collapse probability and an accelerating product roadmap at the same time without hypocrisy. Inside the worldview, the technology is coming regardless of what any individual company does. Given that, the only live question is whether the most safety-conscious people build it first. The probability of doom converts into an obligation to accelerate. This logic is internally coherent, completely unfalsifiable, and perfectly aligned with the believer’s financial interest, three properties that make it extraordinarily stable. I want to be careful here: I am not claiming insincerity. The evidence suggests many of these people genuinely believe what they say. But sincerity shaped by selection is not independent evidence. The colleagues who found the 25 percent intolerable have already left these companies, often loudly. The population remaining is, by construction, the population for whom the logic resolves the tension. I have written before about [vibestatrophe], the way a community’s ambient assumptions stop requiring defense and start functioning as background reality. This is that, operating at the executive level.

The final mechanism is the simplest. These statements cost nothing when they are wrong. Paul Ehrlich predicted hundreds of millions of famine deaths in the 1970s, was wrong about essentially everything, and collected a MacArthur grant and fifty years of respectful interviews. The peak oil forecasters paid no price. Failed apocalyptic predictions have usually, in the modern record, not damaged the predictor. The discourse punishes being boring, not being wrong. And the structure of the claims is built for graceful retreat: probabilities become “I said it was uncertain,” timelines drift outward, and warnings that fail get recast as warnings that worked because vigilance saved us. A statement with unbounded upside and no downside will be repeated. That is not a conspiracy. It is an incentive gradient doing what incentive gradients do.

If these numbers were genuine probability estimates, they would behave like probability estimates. They would visibly update, in both directions, as evidence arrived. Strong capability results should move them up. Three years of falsified short timelines, stable employment data, and models that still require elaborate scaffolding to execute bash commands safely should move them down. Instead the numbers are remarkably stable. The 10 to 25 percent range quoted today is roughly the same as in 2023, when an enormous amount of evidence emerged. A genuine credence moves. A positioning number holds.

So here is the falsifiable prediction this piece stakes: the doom statements will continue at roughly the same probabilities regardless of what the evidence does, because the statements are not tracking evidence. Check back in two years. If the stated probabilities have visibly updated in response to specific named evidence, I am wrong about their function. If they are still 10 to 25 percent, quoted in interviews alongside the same Manhattan Project references, then they were never estimates at all.

None of this means AI carries no risks. It carries real ones, several of which I have written about at length, and the most serious receive a fraction of the attention these statements absorb. It means that when you encounter the genre, the productive question is never how do we survive this. It is the question any genuine estimate can answer and any positioning statement cannot: what, specifically, would change your number?

Sean Richey, Ph.D., is a Professor of Political Science at Georgia State University specializing in AI information environments and digital political communication.

Dr. Richey provides expert witness testimony, case review and analysis for counsel, survey methodology evaluation, and policy consulting on AI-associated information environments. Visit my website or email consulting@seanrichey.com.