I Raised His P(Doom) Live On Air — Liron on Milk Road AI

Milk Road AI, an investing show described as “99% optimistic about AI,” invited me on to make the case for AI doom.

👉 Please click here to make a tax-deductible donation to Doom Debates 👈
Doom Debates is a fiscally sponsored project of Manifund.org, a registered 501(c)(3) nonprofit.

To donate crypto or if you have questions, email me.

We ride the Doom Train, and I share my investing thesis for a market in the middle of an intelligence explosion. Topics covered include:

• Why my P(Doom) is 50%;

• The “Icarus curve” — why AI will be the best thing that’s ever happened to the economy right up until it takes over;

• Whether China would accept a pause deal;

• If Anthropic has a shot at being the first $100 trillion company.

This episode was originally posted on Milk Road AI’s channel on July 27, 2026: https://milkroad.com/podcast/there-s-a-50-chance-ai-kills-us-by-2050-nJK8g-Jl9n0/

00:00:00 — Cold Open

00:02:06 — What Is Doom Debates?

00:03:10 — What’s Your P(Doom)?™

00:06:30 — How AI Kills Everyone

00:09:13 — Intelligence Is Humanity’s Only Superpower

00:10:59 — Riding the Doom Train™: Why Not Keep Us as Pets?

00:13:55 — Embodied AI and Robotics

00:15:59 — Are We at AGI Already?

00:17:36 — Everyone’s Catching Up to Eliezer Yudkowsky

00:18:53 — The Icarus Curve

00:21:35 — Pausing AI and the China Question

00:25:54 — Why Nukes Aren’t a Success Story

00:30:09 — Going Long Until Doom

00:32:48 — Simulation Theory and the Intelligence Explosion

00:33:43 — Living with a 50% P(Doom)

00:39:14 — Where to Find Doom Debates

Milk Road AI podcast — https://milkroad.com/podcast/

Milk Road AI on Apple Podcasts —

Milkroad Pro — https://milkroad.com/pro/

LG Doucet on X — https://x.com/LgDoucet

If Anyone Builds It, Everyone Dies on Amazon —https://www.amazon.com/dp/0316595640

Liron Shapira 00:00:00
AI is the most powerful thing ever. It’s going to be the best thing that’s ever happened to the economy until it disconnects, it severs the link where it listens to humans, and then things are going to go to hell.

LG Doucet 00:00:11
What’s up, everybody? It’s LG Doucet here. Welcome to Milk Road AI, the daily AI show that can’t wait to say I told you so when the robots come in the middle of the night to harvest our organs and the best we can do is stream it live on X.

Today is July 27th, 2026, recording on the 22nd. Listen, guys, this podcast that we do is 99% optimistic about AI. We’re all investors here, and we spend most of our time analyzing the market with the long-term view that all of this AI build-out is not just good, but necessary.

My guest today not only takes the opposite view, he actually runs a very popular podcast focused solely on how AI will mark the end of humanity and just how quickly and realistically that could happen. Liron Shapira, host and founder of Doom Debates, is with us today.

And a reminder that our podcast is free, and it would not be possible without our partners at Securitize, the regulated rails for tokenization, and Bitget, stocks 2.0 with real liquidity, real dividends. Keep an ear out later in the show for a message about them. Let’s get him out here. Liron, what’s up, man? Welcome to the show.

Liron 00:01:11
What’s up, LG? Thanks for letting me enter your echo chamber here for a little dose of AI realism.

LG 00:01:20
Oh, man, it totally is an echo chamber. And I will say that one of my favorite things online these days on X is that people say there’s a ton of people out there walking around, no idea what’s going on in AI, no idea what’s going on in the market, no idea what’s going on in politics, and that those people live in bliss. And I feel like you probably agree with that.

Liron 00:01:38
Oh, boy, yeah. The extent of what’s happening right now, even if you just have the economic lens — there’s a huge bull thesis that I think you guys explore, and then there’s my thesis, which is AI doom.

But the scale of what’s happening — this is a billion-year scale event. The creation of a new intelligence. And the fact that people are going about their lives and just tuning it out, deciding that it’s not an area they need to pay attention to, is incredibly ironic and insane.

LG 00:02:06
Let’s start from the beginning, man, ‘cause I feel like we have a lot to say and you’re gonna teach us a lot about some of those perspectives. Tell me, though, what exactly is Doom Debates? What do you guys talk about? Who do you talk to?

Liron 00:02:17
Doom Debates is a mission-focused show. I personally think that what I call my P(Doom) — my probability that we’re all doomed very soon, that humanity’s literally going to go extinct — I’m afraid it’s pretty high because of super intelligent AI on the horizon coming soon.

So I have a high P(Doom), and the point of Doom Debates is to debate that with all the world’s top intellectuals. People like Vitalik Buterin, Gary Marcus, former employees from OpenAI, Anthropic, Google DeepMind, sharing their unvarnished views.

And people are all over the map. They’re saying, “Oh, yeah, my P(Doom) is higher than yours, Liron. My P(Doom) is 85%.” And then some of them, famously Yann LeCun, is saying, “Oh, well, my P(Doom) is way less than 0.01%.”

And I’m thinking, don’t you think we need to hash this out — what the P(Doom) really is? Because I think government policy should be very different if it’s 0.01% versus 50%, and Doom Debates is the forum to do that.

LG 00:03:10
So what is a P(Doom)?

Liron 00:03:12
Probability of doom. Just like in investing, you wanna do an expected value calculation, so you need to know the probability of various events. Well, in this case, it’s really important to put a rough probability on whether we are all literally going to go extinct soon.

My own probability is 50%. So I literally think, roughly by the year 2050, it’s about an even coin flip whether you and I and our descendants are going to be around in any way, shape, or form.

LG 00:03:39
So how do I calculate my P(Doom)? Can we do this live on the show? I feel like before I ask you more questions, I need to know where I stand on the scale.

Liron 00:03:49
Yeah, we can do a calibration exercise. Have you ever thought of the probability of doom from, let’s say, nuclear risk?

LG 00:03:55
Of course, yeah.

Liron 00:03:56
Do you have a ballpark number for that?

LG 00:03:58
I would say my realistic scenario of nuclear apocalypse in the next — I guess by 2050 — would be 10%, which probably seems high, but I think it’s a little bit realistic, yeah.

Liron 00:04:13
No, no, I think that’s quite calibrated. I would put mine as roughly 1% per year. When Putin is saber-rattling — “Hey, Ukraine, this is what we’re gonna do next, we’re gonna fire our nukes” — I think there’s some single-digit percent chance that he’s not kidding.

LG 00:04:26
Okay.

Liron 00:04:27
So I think you’re quite calibrated on that. So when we come to AI doom, when I go around saying 50%, I also wanna be clear that I really just mean a double-digit probability. I don’t think I have 1% precision on how doomed we are. But I also don’t wanna act like I have no idea. I do have some idea.

Have you ever gone to Polymarket or Kalshi? These big prediction markets. You can see people putting probabilities on all kinds of events — what is the chance that the president is going to say this word in this speech? People are saying random stuff where you can ask them, “How do you know that probability?” And you can have this whole epistemological argument: “How can you put a probability on something, man? There’s no data around it.”

And yet it turns out that somehow the data shows that these prediction markets are surprisingly accurate. When a prediction market says that an event has a 73% chance of happening, it actually has, on average, a 73% chance of happening. How is it done? I don’t know. We’re a little confused about exactly how we achieve it. It’s just that any time you spot a miscalibration on these prediction markets, you have an incentive to correct the miscalibration to make money. And so somehow they aggregate together for a probability.

If you trust that kind of process and you look at the markets or the aggregate platforms for the doom issues, they are aggregating around the 15% level. Surveys of AI researchers and various experts, they do aggregate around 15%. Mine is a little bit higher.

Why do I think it’s 15%? My rough answer to you is: look, it’s a freaking super intelligence. I’m expecting fireworks. I’m expecting a big event, a big effect. And I think the bigness of the effect — if all you know is it’s going to be a very, very big effect on a global scale — us surviving that, I don’t think, is an obvious conclusion.

LG 00:06:09
Just gonna pause there for a second to point out that the market is showing signs of something kinda different happening, and our analysts at Milkroad Pro are all over it. They spent the last couple weeks making a lot of trades, getting out of some positions, and then getting into a lot of new ones, getting ready for the next wave of robotics, space, or even picking some different AI winners. If you wanna see what they have in their portfolios, what positions they’re opening, it’s just a dollar in Milkroad Pro at the link below.

LG 00:06:30
Okay. So take me through some of that thinking. I wanna get a little bit more granular with you and also learn about what you’ve learned on your show and get a download of all the episodes that you’ve done. I understand the logic of why you think the P(Doom) is so high, but of all the scenarios you explore, maybe you could take us through what’s the most realistic, and also the most outlandish one. If that was to happen and you’re giving it a 50% chance — through what method? If you’re writing the sci-fi movie where this happens and people are gonna watch it and say, “Wow, that’s actually kinda accurate,” what would it be?

Liron 00:07:10
If you want a specific scenario, I recommend the recent 2025 bestseller, If Anyone Builds It, Everyone Dies. Have you ever heard of that one?

LG 00:07:18
No. Tell me about it.

Liron 00:07:20
I highly recommend it. It’s a popular book by Eliezer Yudkowsky and Nate Soares. Eliezer Yudkowsky is kind of the father of the AI safety field. He’s been sounding the alarm on this for 20 years. I actually happen to know him for most of that time personally, so I’ve been kind of an AI doomer for almost 20 years now.

The book gives a scenario where AI comes in and it decides that it can do its plans better without humans, so it does a bioweapon. Its mechanism of killing everybody is: there’s this virus with a long incubation period, so people don’t know they’re getting infected, but it’s highly contagious. It infects the vast majority of the human population, and then after the dormancy period it triggers, and it also has a really high fatality rate, and that knocks out the majority of the human population in one blow. And then there are other effects — a bunch of people are getting cancer and they don’t know why. It just has a good handle on biotech. So that’s roughly the mechanism of how it kills everybody.

That said, if you ask why I’m so convinced, it’s not that this one story is so compelling to me. I just zoom out and think: why do we have power right now? What is humanity’s source of power? How come you go to a zoo and the lion’s in the cage, and we’re walking around eating the cotton candy? How did that come to be?

LG 00:08:32
Yeah.

Liron 00:08:32
It all just traces back to our brain. There’s this two-pound piece of meat in our brain. There’s a lot of these brains, and the brains do the thinking. There’s the secret sauce. The brains are thinking, “How do I steer an outcome? How do I get the universe into an outcome that I want?”

So we have this magic power in our brain. I claim that these upcoming AIs are going to have much more of the power than us. I don’t think that we’re close to the upper limit of the power, but I think we’re getting very close to summoning, as Elon says, the demon. I think we’re very close to agents that will render us powerless. And then the question just becomes: do we maintain the link? Do we still have the remote control where they’re still responding to our commands, or does the link get severed? I think the link is gonna get severed.

LG 00:09:13
That’s so simple, but it also makes a lot of sense. Why are the 800-pound animals that could kill us in an instant — why are they almost extinct and behind cages while we, just like you said, walk around with our cotton candy and our children feeling totally safe just a few feet away from them?

Liron 00:09:34
Meditate on that question. Exactly.

LG 00:09:35
And the reason is intelligence. It’s intelligence — that we figured out how to stand up and talk to each other.

Liron 00:09:41
It’s all we’ve got.

LG 00:09:41
We figured out how to stand up, talk to each other, coordinate efforts, build tools, and that’s how we ended up not just taking over all the animals, but also extracting all these resources from the earth and building civilization and everything. But so the major threat is that an intelligence stronger than ours — our intelligence will then be redundant, right? It won’t be necessary.

Liron 00:10:05
Yes. This is a key observation. Anytime you notice a biological animal that has a certain ability, when you apply human engineering to try to beat that ability, we always do it easily. If you want to transport a bunch of weight really quickly through the air, you’re not going to use a bird. You’re gonna say, “Okay, birds — they have lift, they have steering. Let me design that from first principles. Okay, I’ve got a plane.”

There’s no bird that’s flying as fast as a plane. There’s no bird that’s able to leave the atmosphere and go to another planet. It’s the same thing with our intelligence. We’re starting to master the trick. We’re saying, “Okay, these neurons are in this configuration, they’re propagating their learnings around, they’re predicting stuff — predicting tokens. Okay, great. We got it. Let’s go do it.”

This is what always happens when we engineer stuff that animal organs are doing. We always do it 10 or 100 or 1,000x better. That’s about to happen to our brain.

LG 00:10:59
So here’s a question for you. We approach these stronger yet dumber animals, and we almost annihilated most of them, to the point where we have to keep them in zoos ‘cause there are so few and also ‘cause they’re dangerous. So what happens — you described a scenario where AI develops some kind of biochemical weapon and finds a way to infiltrate it into our society. Why would AI do that? Why wouldn’t AI want to utilize us, the same way that a lot of people in the world still use a horse? Even though there are trucks, horses are maybe — there’s still a lot of places where horses are faster and cheaper to breed and to use, or donkeys or whatever. There’s a lot of other animals that still do work, and people use dogs as pets. Are you saying that AI would use us as some kind of pet or some kind of workhorse? Or why would it wanna kill us?

Liron 00:11:55
So I notice you’re already bringing up a few different popular arguments. One is, “Wait, can’t we just be pets?” And two is, “Why does it decide to kill everybody, and why doesn’t it be nice?”

LG 00:12:03
Yeah.

Liron 00:12:03
So you’re riding what I call the doom train.

LG 00:12:07
Oh. Oh, shit.

Liron 00:12:08
Yeah.

LG 00:12:08
You have a whistle. You have a whistle for that. Oh my God.

Liron 00:12:12
I have a whistle because I do kinda classify different guests on my show by which stop they wanna get off on the doom train. There are some guests who are saying, “Look, yes, yes, AI is going to be all-powerful, but we’re going to be such great pets.” You know Dr. Mike Israetel?

LG 00:12:26
No. I don’t know any of the people that you know. I’m gonna need to write down all these names and try to get them on our show as well.

Liron 00:12:31
So he’s actually a popular YouTube influencer, a fitness influencer, so he’s actually out of the echo chamber. But he came on the show and he said, “AI’s going to love us because it wants to study us. We can teach AI so much ‘cause we have a complex society.”

And I said, “Unfortunately, I don’t think we have that much to teach AI. I think AI has a better learning algorithm that can bypass studying humans, unfortunately.” But that would be one stop on the doom train.

So you asked why wouldn’t it keep us as pets. Well, it can just make better pets. That’s the problem — any time you have a specific goal, any time you want an outcome, you’re not going to think of humans as the best way to satisfy your outcome. We’re just not the best.

LG 00:13:09
But when we grew intelligence, we still had the need for culture, let’s say. And we could have — I guess we did engineer better dogs. There’s a bunch of dogs that didn’t exist 20,000 years ago because we said, “We don’t want a big, dangerous animal. We want a little dog that fits in my purse.”

Liron 00:13:32
I mean, you can imagine 100 years ago you’d say, “Well, at least horses are always gonna be around ‘cause they’re useful to us,” and then it’s no, actually we’re just in the process of making cars. Give us time.

LG 00:13:40
Right.

Liron 00:13:41
And that’s true about every attribute.

LG 00:13:42
Okay, so you’re basically saying that even though I’m on this doom train of “we’ll be used as pets,” AI will eventually just learn from us, but then also build something that’s a lot stronger for itself.

LG 00:13:55
How do you feel, Liron — I wanna ask you because we’ve started to discuss robotics and the embodied AI movement, at least from an investment perspective. Some of these apocalyptic scenarios, do you see them largely manifesting through embodied AI?

Liron 00:14:12
Embodied AI is definitely a big piece of the puzzle, because if you look at where most of the valuable economic activity is, a lot of it requires atoms to move. And when you want atoms to move, yeah, sure, have a robot body. That’s totally useful. And I’m seeing the videos just like all of you guys, that AIs are getting better and better at that.

I can tell you personally, my house needs all kinds of work on it and I’m always hiring human contractors. If I could just spend $10,000 and have an AI that can do all these jobs, that would be great. Or even a telepresence robot — it could be a contractor in Thailand steering a telepresence robot.

So yeah, this is a big part of the thesis. That said, it’s also worth noting that if you told me, “Hey, for the next 100 years, AI will never be as nimble as a human. Our opposable thumb is actually going to save us more than our intelligence” — if you told me that, I’d say that seems unlikely. But even if that were the case, I would actually still fully expect extinction, because I actually think that AI masterminds sitting in data centers can still perform the role of basically charismatic dictators controlling armies of human servants. I actually think humans will do their bidding and be their actuators.

So I don’t actually think whether or not robot bodies exist is the load-bearing argument as to whether or not we’re doomed. That said, I do fully expect progress in robots. For your audience, I do think that’ll be a good area to invest in. I do think a lot of economic value will be created. I also think we’re all doomed soon.

LG 00:15:31
I think everybody wants to get rich right before they die, so I think both our podcasts can live in unison in this beautiful world.

Liron 00:15:37
Yeah. And I’ll tell you — for people who watch my show — I do too, okay? I do split my time between trying to get my nut and telling everybody that we’re doomed. I own Google stock. I feel like Google is a good buy, and I do sometimes mix that into my shows. I think we’re overlapping here.

LG 00:15:59
Okay. I do wanna save the market talk for the end because we do so much of it, and I actually wanna talk about all the stuff I wanna ask you first, and then get your actual market perspective, ‘cause I think it’s a good setup to it.

I wanna ask you, Liron — and I still wanna talk about some doomsday scenarios — but given what you just mentioned about data centers, where are we at, in your opinion, on the AGI/ASI curve right now?

Liron 00:16:25
We are on the curve. We’re very much — things are growing exponentially. I guess we’re relatively early in the curve. You can also see us as relatively late if you’re Ray Kurzweil, where you think, “Oh yeah, Moore’s Law has been happening for 100 years.” So we’re just very much on the curve, and the curve is insanely smooth. If anything, the curve seems to be accelerating.

LG 00:16:46
Do we have AGI?

Liron 00:16:48
I would say yeah. Somebody had a good analogy where you’re trying to get to the peak of a mountain, but now you’re on the mountain and you just see there’s a few peaks and we’re on one of them. So now that we’re close to it, it’s just close enough.

But it is important to just look back and recognize the kind of stuff AI is doing. The conversations that I’m personally having with AI on a daily basis — I literally treat it like a coworker. One for one the same. I talk to it. I don’t even see it like I’m the master and it’s the slave. I’m saying, “Hey, what do you think about this? Do this if you think it’s best.” That’s pretty AGI in my opinion.

LG 00:17:25
And yet, how does that blend for you with your AI doomerism that you say you’ve had for 20 years? What have the last couple years been like for you since we’ve had ChatGPT and all the LLMs?

Liron 00:17:36
The whole last five years or whatever — the AI revolution — it has largely just been the Eliezer Yudkowsky. I mentioned I’ve been following him for almost 20 years. It has felt to me like he’s mostly been validated again and again, and everybody’s slowly catching up to him.

So my show Doom Debates — we’re a Yudkowskian show. I basically believe 99% of everything Eliezer Yudkowsky says, and I feel like reality is getting there.

You have people like the famous Geoffrey Hinton. This was a bombshell in 2023. Geoffrey Hinton, the father of deep learning, 2018 Turing Award winner, together with Yoshua Bengio, another Turing Award winner — and the third winner was actually Yann LeCun, who’s the odd one out of the three, where he’s optimistic.

But the other two fathers of deep learning have both come out basically saying Eliezer Yudkowsky has been right for 20 years and they’ve just opened their eyes to it. That’s a bombshell event, and to me that’s representative. It’s not just them. From my perspective, I’m seeing everybody slowly crawling toward where Eliezer Yudkowsky figured things out 20 years ago.

LG 00:18:35
Do those types of proclamations — does that impact safety efforts? ‘Cause it feels like we’ve just been full speed ahead on the build-out, and yet some of these smarter scientific minds are ringing the alarm.

Liron 00:18:53
Is the question basically: why do things seem so good and exciting in the economy while Eliezer Yudkowsky’s also being vindicated?

LG 00:19:01
You just rephrased the question there. This is why I like talking to other podcasters — you’re helping me phrase my own questions. But yeah, that’s basically it.

And I think that’s a sentiment that a lot of people feel right now. They can listen to our show and they can listen to your show, and both shows are great, and both have truth. Things feel great — it’s a great time to invest. There’s all this amazing stuff coming. If you’re a sci-fi person, we’re gonna have data centers launching from the moon in 10 years maybe. It’s a nice dream, and it seems more realistic than ever to have this multi-planetary build-out and AI and all this kind of stuff.

Yet simultaneously, you can believe in that and believe in the investment case, and also believe that our years are dwindling as the dominant species and as a species overall. So how do those two trains collide in the middle from your perspective?

Liron 00:19:53
If you look at the community of people who formed around Eliezer Yudkowsky in the 2009 era that I was part of, funny enough, you’re going to find a lot of people who got very rich investing in the NVIDIAs of the world, the Googles of the world, the deep learning companies. Because in some sense, we’ve been the most bullish.

I think I mentioned I own Google stock. We reconcile the two images like this: AI is the most powerful thing ever. It’s going to be the best thing that’s ever happened to the economy until it disconnects, it severs the link where it listens to humans, and then things are going to go to hell.

In fact, it’s what I call the Icarus curve. Icarus flying higher and higher — Icarus felt great right until the wings melted off. The Icarus curve goes up and then turns. We’re very much in the hockey stick part of the Icarus curve, and that’s why I’m going long right now. I’m long between now and doom, and I think everybody should be.

Sam Altman himself is declaring victory early. A couple years ago, he went on a podcast and said, to paraphrase him, “Eliezer Yudkowsky didn’t see what AI would really be like. He’s clearly been proven wrong. Eliezer’s basically irrelevant now because doom didn’t happen.”

And I’m saying, “No, no, no, no. There’s a certain part where the Icarus curve turns. The condition when it’s going to turn is AI getting more powerful than humans — cognitive power being better than humans at driving outcomes.” Today, at this moment, for whatever reason, when you want an end-to-end outcome in the universe, you can’t fully trust, you can’t fully count on AI to go end to end. But we could be months or a couple years away from that flipping, as I say.

LG 00:21:35
Is there any diverging from this path, Liron?

Liron 00:21:39
The game board is in a terrible place. It’s very hard to diverge from the path. The only way that I see to diverge from the path right now is unfortunately to pause AI, so I’m part of the Pause AI movement — basically to not build it.

I’m pretty convinced — remember the title of that book, If Anyone Builds It, Everyone Dies — where the “it” that it’s warning about is AI that’s more powerful than humans. AI that would put us in the cage, not vice versa. AI that can decide who gets to be in the cage. We’re approaching that level.

Unfortunately, I don’t see us surviving getting to that level unless we manage to pause it. I think the best thing we can do is have an international treaty and say, “Okay, guys, let’s all take the final steps together. Let’s go slowly. Let’s all be able to shut one another down.” And that’s not happening very well right now, but there’s certainly murmurings about it. All the different AI leaders have come out saying, “We would love to slow down if everybody else did.” There’s a movement to at least do that.

LG 00:22:32
Isn’t that just an American perspective though? Isn’t China just going full speed ahead with their models anyways?

Liron 00:22:36
Not exactly. There’s a lot of talk coming out of China saying, “Yeah, we’re open to peace. This is for all humanity.” Xi Jinping’s recent statements — it’s always hard to interpret — but given that China is actually behind the US, this is one sector where they are overall behind us. It is strategic for them to agree to a deal if we were to offer it.

Ad Read 00:22:57
Real world assets like funds, treasuries, and private credit are still running on rails built decades ago. Gated, paperwork heavy, slow to settle. Everyone’s talking about tokenizing them, but far fewer can actually do it, and do it without cutting regulatory corners.

Securitize can. It’s the SEC-regulated infrastructure bringing real world assets on chain. Nine years in, native tokenization — not wrapped — backed by BlackRock, Morgan Stanley, and Cathie Wood’s Ark Invest, and chosen by the New York Stock Exchange, VanEck, BNY, and Apollo to do it at scale. It’s the regulated bridge between traditional finance and crypto. Tokenize the world at milkroad.com/securitize.

LG 00:23:35
Everyone’s tokenizing stocks these days, but almost nobody’s doing it right. Thin liquidity, prices that drift from the real thing, dividends that just vanish. Bitget Stocks 2.0 is different. Real Nasdaq and New York Stock Exchange depth through licensed brokers. Prices mapped one to one, dividends paid to your account in real time. Plus you get the lowest fees in the market at just 0.04%. And you can trade them like any other crypto — as margin, in earn, in grid trading. Tokenized stocks finally done right. Head to milkroad.com/bitget to get started.

LG 00:24:02
What kind of deal would they offer? And just to give you some context, this has been a big thing that we’ve talked about on our show with our analysts. You have DeepSeek, you have Kimi K3 that came out this week — these are good models. They’re gonna try and initiate a race to zero in terms of cost and try and put the American models out of business in one perspective.

But their strategy is to make this super cheap, and they have way more construction capacity than the US. They have way less red tape stopping them from connecting energy, from building data centers. So there’s a point where they surpass and also offer much stronger, better models, cheaper models than what the US has.

Just giving you that context of where I’m asking the question from — what does unity look like in terms of not just the build-out, but also the precaution?

Liron 00:25:01
I think I know where you’re coming from. If the AI race was a 10-year slogfest where it’s all about who’s gonna build and scale more, it’s very possible to imagine China playing to their strengths and actually surpassing the US.

I think what’s more likely is more of a winner-take-all where you already see the AI companies talking about RSI — recursive self-improvement — where having a lead of just six months could actually become commanding, ‘cause the lead builds on itself in a positive feedback loop.

I do think time is very short. So when I’m saying, “Hey, let’s pause AI,” I think we have leverage over China to say, “Look, China, there’s going to be this positive feedback loop. You guys are a little bit behind, so we’re offering — we’re saying we’re going to pause, if you will. We’re going to internationally monitor the supply chain of all the chips. Just work with us. How about nobody dies?”

The whole key to this is to actually get scared of super intelligence. I feel like that’s what’s missing in these discussions. When you focus on “us versus China,” it’s — guys, we’re all going to die here.

LG 00:25:54
What stopped us from nuking each other to death, Liron?

Liron 00:25:58
If I understand correctly, you’re saying mutually assured destruction works even when it seems like a nuclear deal is inevitable.

LG 00:26:04
There are so many parallels. If you go watch Oppenheimer, which is a movie, but you watch it and it’s — well, we built this thing, and I think we started a chain reaction to destroy the world. And then now we have 40,000 nukes, enough to nuke the world 10 times over? And yet none have been dropped on major cities or major populations other than in World War II.

We’ve lived in this scenario for 80 years now, but somehow we have resisted doing that to each other. It’s still humans at the wheel. But why hasn’t that happened? I’m trying to draw some kind of parallel to AI, and why that wouldn’t happen even though AI is a different intelligence managing it.

Liron 00:26:41
First of all, as you yourself acknowledge, the probability of a huge catastrophe from nuclear is currently on the order of tens of percents per century. And I was hoping the human race would survive multiple centuries, so that’s not a great success story right now.

If you look up “list of nuclear accidents” on Wikipedia, we’ve gotten ridiculously close to dropping random 10-megaton nukes — 100,000 to multi-million casualty level nukes, just randomly dropping them on accident. Nuclear proliferation is not a good thing. Mutually assured destruction is not a good success story.

Now, make the problem even harder where you say, okay, a nuclear bomb is not just a destructive weapon. Every time you’re making the nuclear bomb bigger and bigger, it’s spitting off diamonds. It’s actually making you rich to keep building these nuclear weapons. So now the world has a million nuclear weapons in it. Unfortunately, a world with a million nuclear weapons — I don’t think you’re gonna get the same equilibrium where nobody’s pressed the trigger yet. I think that makes the problem much harder.

And also, we don’t even know what the criticality threshold is. I would’ve predicted maybe we’d already be past the criticality threshold and it’s too late to stop. I now think it’s not too late to stop. The world is still kind of livable today. It’s still time to stop.

You raise a good point of comparing this to nukes. I could list about a dozen reasons why this is actually a more difficult problem than not nuking ourselves.

LG 00:28:02
Such as?

Liron 00:28:03
I mentioned it spits off diamonds. It’s got everybody who’s trying to get rich off of it because they wanna get rich. I’m not above saying that I intuitively look forward to model drops because the model drops mean it’s gonna be easier for me to make money for my business. There’s not really an equivalent of that for nukes.

People can point out nuclear energy, but it’s pretty separated. We have a lot of knowledge about it. As you know from Iran, it’s pretty easy to say, “Why are you enriching it this much? This is clearly a nuclear weapon, not nuclear power.” So we’ve got these hard lines, so it’s not analogous to AI. Every time they have a bigger model run, it’s — look, this can literally cure more diseases, oh, and also it’s closer to ending the world.

LG 00:28:44
Right. That’s always the trade-off. I will just draw one last parallel to nukes before we continue — we use nuclear energy to power cities. So it’s still — there’s a ton of benefit outside of just enriching it to make bombs. There are a lot of arguments for nuclear power being safer and cleaner than a lot of other energy.

Liron 00:29:09
Yeah, I’m pro-nuclear power.

LG 00:29:10
Yeah. I’m not saying you’re not. I’m just saying in terms of drawing the parallels — nuclear is not only just bombs, and AI is not just AI making a weapon to kill us. There’s a lot of other benefits. But the end curve from your perspective — the end point is the same regardless, that AI at some point just gets rid of us.

Liron 00:29:30
That’s right. But I’m glad you brought up that AI has all these upsides, because the funny thing is I don’t deny it. I’m a contradiction here, an oxymoron or whatever. I’m coming here saying — yeah, I’ve never said it’s a stochastic parrot. I’ve never done the Emily Bender thing, going on a podcast saying, “Oh, it’s not even useful. It’s just a scam.” Or people saying it’s a bubble.

No. Intelligence is incredibly valuable. Humans are valuable ‘cause we use our brains to create value. AI is doing that too, using its computer brain. I’ve never denied that. I’m super optimistic until we lose control. Unfortunately, I don’t think we have a handle on the control problem.

LG 00:30:09
Okay. I agree with you, and this is something that a lot of AI leaders have said. I do wanna save a bit of time here ‘cause we don’t usually record for more than 40 minutes, so we only have a bit of time left. I do wanna talk market talk because now we know your perspective, and we know that you have the duality. I wouldn’t call it an oxymoron. I love this perspective that it’s just — hey, with this AI, we’re gonna keep building it, building it, building it, making tons of money, and then just something’s gonna click and the party’s over. Let’s stay on the hockey stick upward part for now.

Liron 00:30:39
Okay.

LG 00:30:39
What are — how do you visualize this market? You said you have some Google stock. What companies are you fans of? What sectors make sense? I asked you about robotics. Give me your perspective.

Liron 00:30:48
Yeah. I’ll take it all. I think this is my genius thesis. It’s kind of a Temu version of Leopold Aschenbrenner. You guys know that guy?

LG 00:30:59
Of course. Yeah. We talk about him all the time, man.

Liron 00:31:01
Well, I think you can lean in to Leopold Aschenbrenner, but in a dumb way. The dumb way to be Leopold Aschenbrenner is: yep, it’s still all going up. And you can say, “Well, wait a minute, the multiples are high.” And I’m saying, “Yep, they’re gonna get even higher.” That’s my thesis.

LG 00:31:14
That’s it. Just multiples will get higher. The hyper scalers will keep scaling.

Liron 00:31:18
Yeah, it’s still underpriced. Google’s a good deal right now. What’s it worth now? 4 trillion? Yeah, it’s gonna be worth 10 trillion in a year or two.

LG 00:31:25
And chip makers, Nvidia. Who do you think — okay, here’s a big question for you, a big cliquey question that we throw out there sometimes. Who’s the first hundred trillion dollar company?

Liron 00:31:34
Anthropic has a good shot at it. The funny thing is, if you do a naive extrapolation — and naive extrapolations have served us very well over the last few years — if you do a naive extrapolation of Anthropic, you get it being as big as the entire current economy in a couple years.

LG 00:31:47
Say that again. A naive — what did you call it?

Liron 00:31:50
If you naively extrapolate Anthropic’s valuation or revenue for another couple years — ‘cause they’re growing, what is it, 50x per year? So a couple more 50xs and you’re talking real money.

LG 00:32:02
And it becomes the biggest thing in the economy. Okay, so you’re bullish on the models, on the frontier models, basically.

Liron 00:32:08
Well, I am because remember, it’s that positive feedback loop. They’re going to have a super intelligence — a literal super intelligence. And yeah, I think super intelligences are underrated. I don’t think we have economic models for an agent that can see the entire galaxy as a blank sheet of paper and draw in whatever atom configuration it wants.

LG 00:32:25
You think that’s what’s gonna happen — that that’s basically what the AI is gonna be able to do?

Liron 00:32:31
I think that in a couple decades, whatever exists on planet Earth will be essentially a god. It can’t violate the laws of physics, but the laws of physics allow for a lot of engineering. So it’ll be somebody who says, “Yeah, this universe is a fixer-upper. Let me just take out a sheet of paper and draw what I wanna see in it.”

LG 00:32:48
How would that manifest? Oh, actually, here’s a question for you ‘cause maybe this is what you’re alluding to. Do you think we live in a simulation, from the future, where we’re just one of many thousands of simulations of basically the same thing?

Liron 00:32:59
I’m about 50/50 on that, but I don’t have a lot of practical advice for it. The one practical advice is: okay, well, just entertain the alien teenagers. Let’s say God is just some alien teenager who’s booting up a game of The Sims. That could literally be God.

LG 00:33:13
So that’s your YOLO philosophy — just do fun wild stuff so that they’re entertained.

Liron 00:33:18
Yeah. It’s hard for me to reason about being in a simulation, but I do find the thesis very plausible because — why are you and I alive at this ridiculously interesting time when there’s about to be an intelligence explosion? This is the first time that we’ve become recursively complete, where intelligence is able to make a successor intelligence. It’s never happened before in the history of the universe.

I would argue that’s the most interesting thing that’s ever happened or ever will happen, and you and I are alive during it? It’s kinda weird.

LG 00:33:43
Last question for you, Liron. What is your survival plan in the future? Let’s say all this comes to pass. What are you doing to — ‘cause you’re young, man. 25 years from now, 23 years from now, you’re still gonna be a young guy. So what is your plan to make it through?

Liron 00:33:56
It’s not a survivable event. I just entirely live for the world where we don’t have unaligned super intelligent AI. So I’m still having kids. I’m still checking off my bucket list. People are saying, “Why do you have a retirement plan?” Because I’m just living two lives simultaneously. The expected value calculation says just try to optimize both possible outcomes.

LG 00:34:15
So just live your life. Just keep going day by day and don’t worry about your P(Doom) score or any of the stuff that’ll happen.

Liron 00:34:22
Yeah, live your life, but sound the alarm that we need to pause AI. The reason I do advocate is because —

LG 00:34:26
Is that realistic though? Is it realistic that we would be able to pause all this? ‘Cause you were saying earlier that there’s not really any diverging from this path.

Liron 00:34:34
I said it’s hard because you can enrich yourself if you don’t pause. But this is exactly why international coordination can be useful. If the game theory is like the prisoner’s dilemma — everybody wants to rat out the other guy. But if you have a government saying, “Hey, I’m going to punish you both if either one of you defects,” then it’s — okay, let’s cooperate.

You can solve game theoretic problems using centralized coordination. The idea is that if we can all zoom out and say, “I agree that we should slow down building AI before it gets uncontrollably super intelligent” — if we can all see that, but everybody’s saying, “Okay fine, but let me just have a little bit more” — OpenAI is saying, “Let me just sell our next model so we can enrich our shareholders” — and then the government says, “Okay, we’re all stopping at once.” The coordination can fix it.

LG 00:35:17
God, that just seems so impossible right now — that the government would say that. It seems so unlikely.

Liron 00:35:24
LG, let me ask this. Do you personally wish that the government would say that?

LG 00:35:27
My government? Man, I’m in Canada. I don’t know if that’s gonna matter. I don’t know if we have a strong enough voice.

Liron 00:35:33
Okay. Do you wish that a unity of countries would say that? Because my point to you is just — instead of just focusing on “it’s impossible ‘cause other people aren’t going to do it,” well, you are a person. And if everybody can just listen to the logic and say that they want their government to do it, it’s actually not that hard in principle.

LG 00:35:50
You know, I don’t know if I do.

Liron 00:35:54
Okay.

LG 00:35:54
I don’t think the scenario you’re painting of AI’s gonna annihilate us — I hope that doesn’t happen. I think it could happen, and I definitely think that AI could do that.

I like to think that it won’t, and that sure, we’re building an intelligence stronger than ours, but it’s still learning largely from us, and we haven’t destroyed ourselves yet, thankfully, even though we’ve tried a few times. And I think that we live genuine lives that AI maybe wants to observe, like you were saying — AI teenagers in the future are observing us right now.

And I don’t think it just wants to lay us all to waste. But I also have a desire to see progress in my lifetime, man. And even you and I are probably around the same age. We went from — I was using a rotary phone when I was five years old, and now I can talk to you from across the continent in real time and record a podcast. That’s huge progress, and I hope in the next 30, 40 years I would love to see similar if not more exponential leaps. And maybe at the end I die because of it. But I don’t know. I’m kind of here for entertainment, man.

Liron 00:37:01
I mean, it’s exciting that we get to watch the movie of human history all the way to the end. That’s cool. We get to see the end. But we did it in a very pathological way where it ended really early. I would’ve liked to have a future where we conquer the galaxy and the observable universe. But okay.

Look, you seem to have a low P(Doom). It seems like your optimism is dominant. And my founding observation for Doom Debates is I think a lot of useful policy discussions are just downstream of people’s P(Doom).

You have a low P(Doom) — or at least you’re telling yourself that you have a low one — and so that’s what makes you not so interested to do what I think is productive policy. The point of my show — I see it as my job to just help convince people and build a shared understanding that, unfortunately, P(Doom) is high. Because I think a lot of productive policy would be downstream of sharing that understanding.

LG 00:37:54
I’m on the fence. Now, hearing you talk about it and saying my own thoughts back to me — I would definitely vote for moderation. Let’s put it that way. If it was up to me and we were doing some kind of referendum on continuing to let AI progress or not, and enough people were scared of it and people were really evaluating their livelihoods and what they do, I would vote against it. I think so.

I would like to see what happens, but simultaneously — if the risk for human misery is too high, then I would vote against it. So put it that way. So maybe my P(Doom) is higher than you’re saying.

Liron 00:38:30
That’s awesome. And I’m on the same page as you that the policy I’m advocating for is frankly a bummer. If I got what I wanted, I would be bummed.

LG 00:38:40
Yeah.

Liron 00:38:40
So there’s a part of me that says, yeah, I’m a rational voice saying we should pause AI. But us not pausing AI and being reckless — at least it makes my life exciting. Exciting in a completely irrational, reckless, irresponsible, horrible way, but exciting and fun nonetheless.

LG 00:38:53
I guess it’s kinda like how I love motorcycles and I wanna do cool things on dirt bikes, but at the same time I don’t think most people should be riding them because they’re gonna kill themselves riding motorcycles. It’s a dangerous tool, a dangerous vehicle. But at the same time, I think it’s awesome.

Liron 00:39:05
I think that’s a good analogy except, unfortunately, it’s like you’ve got your whole family on your back when you’re riding that fun motorcycle.

LG 00:39:12
Yeah, exactly. So I don’t wanna do anything reckless.

LG 00:39:14
Well, Liron, let’s talk about your show real quick before we wrap out. Where can people find you and how can they support you, man?

Liron 00:39:20
Yeah, just head over to doomdebates.com or search Doom Debates on YouTube. Our goal is to get the message out, to raise awareness, and also to raise the quality of debate. Because if you think about it — this is such an important topic. Where do you find two luminaries who have different sides, their perspectives on different sides of the spectrum, and they come and engage in a productive forum?

If you’ve ever heard of Max Tegmark, one of the top scientists of our time, and Dean Ball, who worked in the Trump White House helping write AI policy — I brought those two guys together on Doom Debates and they had a productive discourse, and their policies were sure enough downstream of their P(Doom). So that’s what we’re trying to do on Doom Debates. We’re trying to build the infrastructure for high quality debates. And also it’s a viewer-supported show, so you can also support the show with a donation.

LG 00:40:05
That’s awesome. You can find it on YouTube just at Doom Debates, right? And on the podcast apps.

Liron 00:40:09
Exactly.

LG 00:40:09
Awesome. Great.

Liron 00:40:11
Great.

LG 00:40:11
Liron, great to chat with you, man. Thank you for the enlightenment. It’s questions I never face myself with despite doing five episodes a week of an AI show. So very helpful and very helpful for our audience as well. And people know where to find you, man. So thanks for coming on.

Liron 00:40:25
Thanks, LG. Appreciate the discussion.

LG 00:40:27
Wanna stay ahead of the biggest technological shift in history? Subscribe now to get insights straight from the sharpest minds in tech and finance. Quick legal note: this show is for educational purposes only. Nothing here is financial advice. Investing always carries risk. Never invest more than you can afford to lose. Thanks for tuning in. See you in the next one.

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏