#46 - A Hell of a Kill Switch

On the 9th of June, Anthropic launched Fable 5 and Mythos 5, the two most capable models it had ever shipped. Three days later, on the 12th, the US Commerce Department issued an export-control directive ordering Anthropic to cut off access for any foreign national, anywhere, employees included.

The only way to comply was for Anthropic to yank access from everyone, globally. As I write this, two and a half weeks on, Fable 5 is still dark. Mythos 5 came back on the 27th, but only after Commerce handed Anthropic a list of roughly 100 approved US organisations cleared to switch it on again: the government taketh away, the government giveth (maybe).

I managed to play around with Fable 5 before it got pulled. It produced this Elden Ring inspired 3D game in one shot. Not bad.

The reason this all happened is that the US government became aware of a “narrow, non-universal jailbreak” that, by Anthropic’s own account, is equivalent to asking the model to read a codebase and fix the security flaws it finds. Anthropic of course disagrees that a narrow jailbreak justifies recalling a model deployed to millions of people. That didn’t matter.

At the same time, risk professionals in enterprises are talking about the need for a kill switch for AI agents in order to better govern them. Meanwhile the US Government just proved export controls are a hell of a kill switch. Ouch.

People are... unhappy. On the 23rd, Legion LegalTech, a San Jose outfit that builds drafting and case-management tooling on top of Anthropic’s models, sued the US government in DC federal court to get the order vacated and the enforcement enjoined. The grievance: a US company, hit by a US export control aimed at foreign nationals, because its development team happens to sit in Canada. In their own words, here’s how Legion describes the damage: “immediate, irreparable, and existential”. Hyperbole? Maybe. But when a lawsuit becomes part of your business continuity plan, you know something’s up. (Reuters)

when a lawsuit becomes part of your business continuity plan, you know something’s up.

This is also not about just Anthropic. On the 26th, OpenAI shipped GPT-5.6 under precisely that logic: a limited preview to roughly 20 pre-approved organisations, via the API and Codex only, with wider release deferred at the government’s request. Altman told staff it was “not our preferred long term model”, which, just as with Anthropic, went nowhere. The stated trigger was raw capability as opposed to any concrete incident. A source told Axios the government stepped in because GPT-5.6 has “Mythos-like” capability.

As providers cross this capability-line the model release becomes a government-gated rollout automatically. It all sits on top of an AI-security executive order signed on the 2nd, standing up a testing protocol labs are meant to clear before they ship. That protocol is voluntary. OpenAI went with it, probably because they didn’t want an export control imposed on them as well. So much for voluntary. One blackout is an incident. Two labs and a standing review framework is a regime, and who ships, when, and to whom now carries a government stamp.

A frontier model isn’t infrastructure. It’s a service a government can switch off in 72 hours, then decide who’s allowed to plug back in.

I’m guilty of this. I’ve been caught saying AI models are infrastructure more than once: always on, fungible to an extent and it’s someone else’s problem: you pay the bill but don’t run the power station. Take electricity as an example. It has a century of regulation, redundancy, and grid interconnects behind it so that no single failure takes you down. Frontier labs have none of that.

This is also interesting because companies started picking sides: “We’re a Claude shop”. “We standardised on GPT”. “We’re a Google house”. They’re all single points of failure. We’d never accept a single-AZ design for anything that matters. But if these products rely on a frontier lab that can pull access at any point, what happens to your systems and customers? (and no, load-balancing between Anthropic and Bedrock would not help you here)

Before, you might have said that this is the US government getting back at Anthropic. Now that OpenAI joined the fun, we know we should treat this with the right level of care.

Back in March, in Why Guardrails Don’t Work, I argued that bulletproof guardrails aren’t a thing you’ll get to to next sprint: you cannot prove that a sufficiently capable model has no jailbreak. Just as with automated testing, guardrails can’t prove the absence of a jailbreak, only its presence.

The White House is now demanding precisely that proof from Anthropic as the price of switching Fable 5 back on, and the reporting is fairly blunt that it may simply not be achievable. The demand can’t be met, so the model stays offline.

And six days before the OpenAI news broke I posted about the second-order consequence:

“when the dust settles, I personally won’t have any confidence frontier labs won’t withhold their best research from here on.”

I take no pleasure in being right about this (well, I do a little). If shipping your best model means it can be recalled the same week, held back for weeks of government review, and then rationed out to a handful of approved partners, labs might ship less, ship later, and keep the sharpest work behind closed doors. So the closed frontier gets both riskier under this regime as well as slower and scarcer. This changes the calculus on what you’re actually renting when you wire your whole product to it.

While this all unfolds, an open-weights model from China’s Z.ai (formerly Zhipu) called GLM-5.2 posted a 62.1 on SWE-bench Pro, beating GPT-5.5 on long-horizon tasks, at roughly one-sixth the API cost. The model’s weights have been released under an MIT licence, which means you can pull them off Hugging Face and run them on your own hardware.

Now, there’s a real caveat here: using GLM-5.2 via Z.ai’s hosted API carries a China-data-residency question that any regulated org needs to consider. But you have options: pull the weights and run it in your infrastructure, or find an inference provider with different a data-residency story (e.g.: DeepInfra offers US inference). But that caveat is precisely the argument for the open-weights version.

For years “open weights” got filed under downgrade, the thing you use if you couldn’t afford frontier. The “you must pay frontier-API prices” assumption no longer holds, and when an open-weight model is providing frontier-level capability for tasks you care about, can you really afford to ignore non-US open-weight models? This is as much about cost as it is about risk.

Open weights stopped being the thing you settle for. It became the thing that can’t be switched off.

In the everlasting words of Linus Torvalds: Talk is cheap. Show me the code.

I run Liv4All, a personal AI-assistant platform I’ve written about before. It’s been leaning on Claude Sonnet 4.6 for its core reasoning. A couple of weeks ago I started trialling GLM-5.2 in Sonnet’s place. The result, with the usual disclaimer that this is my workload and my observation window: roughly 50% cheaper, with no performance degradation I can discernibly point to so far. (as demonstrated by daily use and my internal evals)

This doesn’t prove GLM-5.2 is “as good as Sonnet” in the abstract. That’s not a useful question without specifying the task. But it shows that, on a real, running, multi-step assistant workload, a model I could in principle self-host did the job at half the price and I’m yet to find a gap. That’s the whole point. My experiment was driven primarily by cost constraints. But I ended up buying myself some redundancy along the way.

And because Liv4All runs behind an abstraction layer, that swap was a config change. Which is the only reason I could run the experiment at all. The teams that hard-wired their LLM provider can’t even test the alternative, let alone switch to it. (but you haven’t done that right? right?).

The blackout will probably end. Mythos 5 is already trickling back, and Fable 5 will probably follow soon, on whatever terms the government sets. What matters is that, in two and a half weeks, “what’s our plan if the model disappears?” went from a paranoid scenario to a live operational question, and most orgs found out their answer was we don’t have one.

At its core, this is about three assumptions we made and shouldn’t have:

Sovereignty. You don’t control the model, where it runs, or who’s cleared to use it.

Redundancy. You concentrated your most critical dependency in a single provider whose availability you can’t guarantee.

Trust. Partnering with a frontier lab buys you a real research superpower. It does not buy you any guarantee you’ll keep getting their best work.

I’ll keep using Claude and GPT. And I think you should, too. They are excellent models. But don’t bet the whole product on a capability you can’t keep, can’t inspect, and can’t switch back on yourself. Own some weights. Run the experiment I ran. Find out what the premium was actually buying you.

It turns out there was a kill switch within your closed-source models after all.