Most AI governance debates start from a comforting premise:
If we slow technological progress, we reduce existential risk.
Philip Trammell (Stanford) and Leopold Aschenbrenner (Situational Awareness LP) challenge that premise with a clean economic model that forces a harder question: what if stagnation is not safe?
Because if dangerous capabilities already exist, slowing progress doesn’t eliminate the risk; it just prolongs the time we remain under today’s level of danger, delays the transition to a safer regime, and raises the cumulative odds of a catastrophic failure.
That flips the default policy intuition.
This article translates this framework into decision-grade implications for leaders, investors, and anyone allocating capital under regulatory uncertainty.
Much of the existing economic literature effectively assumes stagnation is perfectly safe.
That assumption matters more than people realize.
If stagnation is safe, then “go slower” is always at least directionally attractive: you avoid risky experiments, risky deployments, and risky surprises.
But if stagnation is not safe, because some technologies already impose a positive probability of catastrophe each year, then slowing down can be the opposite of caution.
It can be a commitment to run the same risk forever, which raises the cumulative odds of an existential failure.
Existential risk can be treated like a hazard rate: a background probability that catastrophe happens in any given period of time.
Then survival becomes a simple concept:
Your chances of making it long-term depend on the total amount of time you spend living under danger.
Not just how scary things look today. Not whether risk spikes in one moment. What matters is the cumulative exposure.
Two implications follow immediately:
Survival isn’t driven by whether risk spikes in a particular moment. It’s driven by cumulative exposure - the total amount of time you spend living under danger. That’s why accumulation matters: even a small, steady risk becomes a large risk if you live with it for long enough.
And if the hazard never goes away, if it stays above zero indefinitely, then the long-run logic gets grim. Over a long enough timeline, the probability of eventually getting hit trends toward certainty. You don’t need a single “worst year.” You just need enough years.
That’s the intuitive point:
If you keep rolling the dice forever, eventually you lose.
And that’s why “slowing down” can be counterproductive when dangerous capabilities already exist: it can mean more time living inside the hazard, not less.
The paper’s most useful move is separating two different kinds of danger that people keep collapsing into one word: “AI risk.”
This is the danger that comes from capabilities already in the world, even if progress stops tomorrow.
If nuclear weapons exist, there’s ongoing risk from accidents, escalation, theft, and misuse. If advanced bio tools exist, there’s ongoing risk from leaks and bad actors. If powerful AI systems are already deployed, there’s ongoing risk from misuse, errors, and knock-on effects.
The key feature: state risk persists during stagnation. If you freeze progress, you don’t freeze the hazard. You just keep living inside it.
This is the danger created by the process of advancing:
Training bigger models. Running risky experiments. Deploying new systems at speed. Scaling infrastructure and usage faster than governance can adapt.
The key feature: transition risk is tied to momentum. If you slow down, you may reduce the number of risky “frontier moves” happening per year.
Most debates treat all of this as one blob, and then jump straight to “so we should slow down.”
But the policy conclusion depends entirely on which risk dominates:
If state risk is large, slowing down can make you worse off by keeping you in a dangerous world for longer.
If transition risk is large (and gets much worse when you move fast), slowing down might genuinely help.
So the real question isn’t “is AI risky?” It’s where the existential risk comes from: the dangerous world we’re already in, or the process of trying to move beyond it.
In the “state risk only” case, there is a clean result:
When hazard depends only on the technology state, accelerating through a range of states reduces cumulative risk because you spend less time exposed at each hazardous level. Formally, the change in cumulative risk from acceleration is weakly negative. Acceleration is always risk-minimizing.
What’s important here is the strategic meaning:
If you are already in a dangerous world, your default options are not “safe” vs “risky.” Your options are often:
Move faster toward a potentially safer regime
Or remain longer inside a hazardous regime
That puts the burden of proof on broad slowdown advocates: they must show that moving faster increases risk enough to offset the reduced exposure time.
Also, slowdown is never global.
Even if slowing down would reduce risk in a single-actor model, the real world is multi-actor.
A “pause” only reduces transition risk if it is broadly coordinated and enforceable. Otherwise, unilateral slowdown just shifts the frontier to whoever keeps building, often with weaker safety culture, less oversight, or different incentives.
You don’t eliminate the hazard; you lose influence over how the technology is developed and deployed, while the capability still arrives - just under a different governance regime.
In practice, that means slowdown can actually increase strategic and tail risk: you stay exposed to existing dangers longer and you risk being overtaken by actors who may take bigger shortcuts.
When society expects faster growth, the future becomes more valuable in today’s terms. That changes the optimal trade-off to regulators. It becomes rational to give up more consumption and convenience now, through stricter standards, enforcement, and safety investment, because what you’re protecting is larger.
This also pulls policy forward in time. If capability is expected to arrive sooner, the cost of waiting rises: guardrails take time to build, and a failure in a fast-scaling world is more damaging (and more likely to trigger a harsh backlash). So expectations of acceleration can motivate tighter policy before the acceleration fully materializes.
If markets believe acceleration is coming, policy may tighten earlier, not later.
That’s a strategic insight for founders and investors: expectations about the speed of AI progress can change the regulatory path today because they change urgency and downside.
If capability is expected to arrive faster, the window to build guardrails shrinks, and the cost of a major failure rises (a serious incident can trigger abrupt backlash and much harsher rules). So regulators move sooner to set standards, build oversight capacity, and define liability.
In practice, a credible acceleration narrative pulls regulation forward in time, even before the technology fully arrives - because institutions are trying to get ahead of the risk and avoid being forced into a panicked, heavy-handed response later.
State risk indicators
Persistent hazard even if progress paused (e.g., existing capabilities, stockpiles, widely diffused know-how)
Risk driven by misuse, accidents, escalation, or coordination failures in a steady-state world
Transition risk indicators
Risk spikes tied to scaling intensity, experimentation concurrency, or deployment velocity
Safety failures are strongly correlated with pace and operational tempo
Slowing down only starts to look clearly safer if risk doesn’t just rise with speed, but rises faster than speed - meaning each extra unit of pace creates more than an extra unit of danger.
That kind of “nonlinear” risk is plausible when:
Failures become correlated at high tempo. One isolated mistake is survivable, but many mistakes happening at once (or cascading) is not.
Governance hits capacity limits. Oversight doesn’t scale smoothly - reviewers, auditors, and safety teams can’t double output overnight, so quality drops as pace increases.
Response systems saturate. Incident response, monitoring, enforcement, and legal remediation all have hard throughput constraints; beyond a certain speed, issues pile up faster than they can be addressed.
Ask yourself: does slow-moving regulation make capital allocation more fragile?
When institutions update slowly, you get regime uncertainty followed by discontinuous jumps (a court decision, an agency action, a headline-driven bill).
That matters because capital hates two things: irreversibility and policy whiplash.
In a low-throughput regulatory environment, the rational move for founders and investors is to avoid big irreversible bets until the perimeter is clearer, or to build with maximum optionality (modular deployments, jurisdictional flexibility, compliance-by-design, contracts that can be rewritten).
The risk here is that the rules arrive late and land hard, turning today’s growth plan into tomorrow’s stranded asset.
When institutions adapt quickly, regulation becomes a predictable constraint rather than a random shock.
That lowers the discount rate on long-duration projects, pulls forward investment, and rewards operators who can meet standards early.
In that world, acceleration can be net-positive because clarity compresses uncertainty, and uncertainty is often the real limiter on scaling.
If most of the danger comes from what already exists (state risk), then a broad slowdown is hard to defend as a safety strategy.
You’re not removing the hazard; you’re extending the time you stay exposed to it.
In that world, the least-regret stance is to keep progress moving while pushing for targeted controls that reduce the hazard without freezing the frontier.
If most of the danger comes from moving too fast (transition risk), especially because institutions can’t keep up (policy frictions), then “full throttle” can be reckless.
The least-regret stance becomes: avoid extreme speed that overwhelms governance capacity, but still maintain some forward progress, because stopping entirely keeps you stuck with the background hazards you already have and risks ceding the frontier to competitors who keep building anyway, often under worse incentives and weaker oversight.
If anticipated acceleration raises the value of the future, policy shifts toward more safety spending even before society gets richer.
Translated: in a high-acceleration regime, regulatory and compliance infrastructure becomes a growth industry, not a tax.
Investable surfaces include:
Tooling that increases auditability, monitoring, and incident response throughput
Infrastructure that makes compliance cheaper per unit of capability deployed
Safety engineering that scales with deployment (reducing convexity in transition risk)
The Aschenbrenner/Trammell frame implies two kinds of infrastructure matter:
Capability infrastructure (compute, energy, chips, data centers)
Safety-and-governance infrastructure (measurement, controls, verification, enforcement tooling)
If state risk is material, and speed is the escape route, then the winning capital allocators fund the ability to move quickly without compounding transition risk. In practice, that means funding speed with guardrails:
See issues early: monitoring, evaluations, and red-teaming that run continuously
Limit blast radius: sandboxing, staged rollouts, and fast rollback mechanisms
Scale oversight: repeatable checks and governance processes that keep pace with deployment
Stay regulation-proof: audit-ready systems and modular compliance across jurisdictions
The paper’s key move is to treat policy as an input into the real-world hazard rate.
In other words, regulation doesn’t just change compliance costs at the margin.
It changes the probability distribution of outcomes by shaping what gets built, how it gets deployed, and how quickly unsafe behavior is detected and punished.
Markets often misprice this because they model regulation like a headline event: a bill passes, a stock dips, the news cycle moves on.
But in reality, regulation is a structural variable that affects long-run tail risk, adoption curves, liability, and the odds of a regime-breaking incident that triggers a clampdown.
When you treat regulation as theater, you model it as a short-term headline shock. But regulation actually changes the rules of the game - adoption, liability, enforcement, and the probability of a regime-breaking incident that triggers a clampdown.
If you ignore that, you underestimate tail risk and overpay for growth that only exists if today’s permissive environment stays in place.
Build “regulatory throughput” as a core capability. If governance capacity is a bottleneck, it becomes transition risk. Don’t let your scaling speed be the reason risk starts rising non-linearly.
Don’t let “broad slowdown” be your safety plan. If your domain has meaningful state risk, delaying safer regimes can increase cumulative risk. Safety strategy must be targeted: constrain the risky transitions, not progress itself.
Treat safety spend like capex, not PR. As the future becomes more valuable, “sacrificing” more today becomes rational. That is a capital allocation mindset, not a communications mindset.
Underwrite the “safety share” rising. If acceleration increases expected future value, policy and private actors rationally spend more on safety sooner. That shifts who captures margin in the stack.
If the catastrophic tail grows super-linearly with speed, the best businesses are those that make safety scale linearly (or better) with deployment.
Scenario-test with state vs transition decomposition. Your base case should explicitly state: What part of risk persists even if progress slows (state) and what part is tempo-driven (transition). If you can’t say that clearly, you’re guessing.
If the hazard rate is positive and persistent, stagnation is not the safe baseline. It’s simply more time spent under the same danger.
Once you accept that, “slow down” stops being a default posture. It becomes a conditional tool, justified only when the act of moving faster creates disproportionate additional risk.
The better objective is straightforward: minimize cumulative exposure. Do that with targeted risk reduction and governance capacity that scales with technological speed.
Avoid strategies that feel cautious but, in practice, only extend the amount of time we remain inside hazardous regimes.
𝘈𝘯𝘺 𝘷𝘪𝘦𝘸𝘴 𝘰𝘳 𝘴𝘵𝘢𝘵𝘦𝘮𝘦𝘯𝘵𝘴 𝘦𝘹𝘱𝘳𝘦𝘴𝘴𝘦𝘥 𝘢𝘳𝘦 𝘮𝘪𝘯𝘦 𝘢𝘯𝘥 𝘯𝘰𝘵 𝘵𝘩𝘰𝘴𝘦 𝘰𝘧 𝘮𝘺 𝘦𝘮𝘱𝘭𝘰𝘺𝘦𝘳
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.