COMPUTE/COMPETE #7

To comprehend the growing geopolitical implications of Artificial Intelligence, one need look no further than Anthropic’s recent release of its latest model—Fable 5—and its immediate blacklisting by the U.S. government. Yet, in recounting the events, leading analyses often choose and hyperfocus on only one particular motive: the decision as a pursuit of AI Safety, an economic moat, or political attack.

Yet, rather than being solely due to one reason or another, the series of events reflects the complexity of an AI ecosystem in which all three narratives serve as both motivation and foil, often at the same time. To focus on only one strand is to miss the broader picture—a mosaic of institutional incentives, personal motives, and structural limits. To understand the decisions made by lab, business, and government leaders, one must first understand these narratives.

Enjoy!

‘There are decades where nothing happens; and there are weeks where decades happen.’

In this potentially misattributed quote, Vladimir Lenin attempted to convey the significance of history’s last great World System theory. To many at the time, Communism, the declaration that history was driven by material conditions in cascading waves of class conflict from Feudalism to Capitalism and, eventually, Communism, amounted to nothing short of a philosophical revolution—one that would, and certainly did, inspire in the mind of humanity a reshaping of the Earth itself.

For some, we have reached a point where that sentence is once again apt. The difference is, this time, the revolution is technological, and it is the dawn of Artificial Intelligence. Demis Hassabis, the CEO of Google’s AI lab, recently declared that we are currently standing in the ‘foothills of the singularity’, or, shortly put, human-surpassing intelligence. What inspired the declaration was his discovery of the first traces of AI’s recursive self-improvement—the ability for models to autonomously develop their own successors. Separately, a panel of experts attempting to rank the technology according to its potential social impact judged it likely to be the ‘technology of the century’, similar to electricity. Yet, unconvinced, they also assigned it a one-in-four chance of becoming the ‘technology of the millennium’, comparable to the agricultural revolution and just under the rise of humans ourselves. One intelligence is biological, and the other man-made, growing in capability, and much more easily reproducible.

But, at this moment, we are still in the foothills, with the path forward unclear and long—by change, not by time. Events from the past week—principally the release of Anthropic’s Fable model and the subsequent decision by the U.S. government to de facto blacklist it—carry disproportionate implications for the road ahead.

What this week shows is that the next few years of AI development will revolve around three dominant and recurring narratives. In many events, all three will be equally true at the same time. Individually, each will drive and be used to justify ever more important decisions by lab leaders, businesses, and governments, and yet some analyses hyper-focus on only one, crucially missing the broader picture.

The first is a focus on AI Safety—that the mismatch of rapidly accelerating models with the insufficient ability to understand or regulate their decisions carries an existential threat to humanity. The threat is individual and institutional: unregulated super-intelligent models may lower the barrier for individuals to launch devastating biological and cybersecurity attacks, while institutions move too slowly to consider the potential societal impacts from the technology, such as the revaluation of labour.

The second consists of accusations of anti-competitive economic practises, ripe with worries of the growing leverage of frontier labs to cement their incumbent advantage and extract undue rent. Central to these fears is the world’s experience with the past three decades of the platform economy, in which digital platforms like Google and Facebook lean on the power of network effects to overwhelm competition with value through size. This narrative asks: Will centralisation be endemic to AI?

Finally, the third is a concern that access to the leading models and the compute to run them will and has already become a political cudgel to be aimed at a country’s national security and economic health. Whether leveraged by labs seeking to nudge a state’s policies and regulatory agenda or by one government desiring to threaten or harm another, AI will become political.

To focus on only one strand is to miss the broader picture—a mosaic of institutional incentives, personal motives, and structural limits. Here is how this story is already unfolding, told with all three:

Trouble has been brewing in the AI world for some time now. Back in March, a fight erupted between Anthropic, the world’s leading lab, and the U.S. Department of Defense after the DoD demanded uninhibited use of the lab’s AI technology. Anthropic staunchly refused, citing an internal promise not to permit its models to be used for autonomous weapons or domestic surveillance until government regulation and model capability matured. The DoD, in response, designated the lab a supply chain risk, a formal declaration that prevents contractors from using Anthropic’s models in their work with the Pentagon, and one traditionally reserved for companies with ties to adversarial militaries like China’s People’s Liberation Army.

The crux of the matter was, and will certainly continue to be going forward, the near-sovereign ability of AI labs to instill certain values and morals into their models during the models’ process of pre- and post-training. These stages are when the models learn how to ‘think’. Anthropic, for its models, created an 80-page Constitution filled with language instructing the models to be broadly safe, ethical, and helpful, with ethical being defined as ‘a good, wise, and virtuous agent, exhibiting skill, judgement, nuance, and sensitivity in handling real-world decision-making, including in the context of moral uncertainty and disagreement’. With the Constitution as a North Star, the models may, and Anthropic certainly wishes they will, independently refuse certain requests, such as when an individual asks how to make an illicit substance, or uses it to design and conduct mass-surveillance.

For the labs, this step is a critical and necessary part of pursuing AI Safety, a broad goal defined as anything that ensures ‘AI technologies are designed and used in a way that benefits humanity and minimises any potential harm or negative outcomes’. What harms could there be? On the smaller scale, an algorithm trained on skewed or misleading data could exhibit biases, which, when applied to automatic decision-making or chatbot responses, could favour some groups or harm others. On the larger scale, an AI model that isn’t bent towards the good of humanity may, if it eventually surpasses human intelligence, pose an existential risk to humanity itself. For the labs building what they consider to be the coming ‘Machine God’, the alignment of models with morally good values (simply called a model’s alignment) is a prerequisite for humanity’s survival and continued agency when artificial intelligence becomes abundant, diffuse, and highly capable. Watching interviews with the leaders running the largest AI labs, one can see this issue personally resonates and is often top of mind, second to perhaps only the question of how to improve the models themselves.

For the state, however, an AI lab’s pursuit of AI Safety through aligning the models to certain actions is an impermissible overturning of the natural hierarchy of things, so far as servicing the government is concerned. According to the social contract, especially in democratic societies, the state is imbued with a principal sovereignty, such as the right to maintain order, enforce laws, and govern its citizens as the highest power. Of course, in a liberal society, there are actions the state is not permitted to take, such as something that violates its constitution, but that line is judged by the state itself. For a private entity, a business especially, to place limits on what the state can do, even if those limits are believed to be moral, erodes its sovereignty and shifts power to unrepresentative forces.

This matter takes on supercharged implications when one considers the question on a global stage. Faced with this limitation, the U.S. government wielded domestic laws to challenge Anthropic and turned to alternative model providers. Now, consider this situation from the perspective of a foreign state, such as Ecuador. First of all, beyond the usual limitations of an aligned model, a foreign state also carries the risk of losing model access if it becomes sanctioned by the U.S. government. With a relatively small market and unable to reach Anthropic’s U.S. listing, Ecuador has little recourse to challenge or punish the company if it places a restriction on how it can use its models. Ecuador could, of course, look for alternative model providers, but, debatedly, all of the frontier models are American as well. There are smaller, bespoke providers like Canada’s Cohere or France’s Mistral, as well as several Chinese open-source models, but these are all widely acknowledged as a performance tier below the American frontrunners. As such, Ecuador must then choose between sovereignty and the best capabilities for its citizens. This circumstance, however, is nothing particularly new, as tech companies like Meta, Google, and Microsoft have long been called ‘digital sovereigns’, impacting the realm of possibility for many states.

Flash forward to the April announcement of Anthropic’s newest model: Mythos. Claiming this latest generation of AI technology, having achieved a hallmark level of cybersecurity capabilities, posed a fundamental threat to ‘every major operating system and web browser’, the lab chose to postpone the public release of the model. Instead, it announced an initiative called Project Glasswing, providing the technology first to major providers of digital infrastructure. On the lab’s part, this decision reflected its commitment to AI Safety due to the ability of the model to supercharge offensive hacking capabilities. To them, it is only right to first allow banks, cloud giants, and cybersecurity firms, among some government partners, to use the technology to shore up their defences before releasing it to the broader public.

One of the critical aspects of this plan, however, was that Anthropic, a private company, played the ultimate decision-making role in choosing which organisations were given access to the frontier model, and thus deciding who would have the chance to strengthen their protections. It did not take long for many to realise the chosen partners were almost exclusively American, with even most U.S.-allied governments not receiving access. It was not until two months later, following the European Commission—a political body—repeatedly petitioning the lab, that Project Glasswing was expanded to cover 150 organisations across more than 15 countries.

Yet, just two weeks later, Anthropic released Fable 5, a public version of Mythos. According to the lab, Fable 5 had all of the same capabilities of Mythos, with one catch: classifiers. Staunch in its belief that Mythos was too powerful in certain fields, Anthropic created an infrastructure of safeguards that would remain alert to Fable 5 users’ queries and be able to detect certain malevolent ones. These classifiers would then prevent the model from responding, instead throttling the response down to a less intelligent model, such as Opus 4.8. Effectively, the lab created a system to decide which questions were permitted certain levels of intelligence.

What types of queries would trigger a classifier’s response? According to Anthropic’s report, primarily jailbreak attempts—when an individual tries to break through the model’s safeguards, as well as requests related to cybersecurity, biology, chemistry, or distillation. The first few are self-explanatory: Anthropic wishes to prevent the model from being used to assist in potentially dangerous activities such as cyberattacks. However, for the last case, distillation—or using the responses of a more intelligent model to as data to train one’s own model, i.e., making the newer model mimic the behaviour of the superior model, the lab asserted its concern with ‘accelerating other AI developers in building powerful AI systems that pose similar risks to the ones ours pose—without necessarily having commensurate safeguards’. To put it bluntly, Anthropic doesn’t trust other labs to meaningfully pursue AI Safety.

Beyond the obvious political concerns with the decision to implement classifiers to models being used by governments, or the newly established precedent to throttle the allotted intelligence of a response, some commentators have also pointed to the monopolistic implications set by this framework. One prominent open-source proponent, Nathan Lambert, argued that the decision to broadly throttle attempts to use Fable 5 to train other models constituted the erection of a capability moat and a declaration of war against competition. He admits Anthropic is within its rights to take such action, but asserts it will harm the broader AI ecosystem, entrenching the company’s lead behind a veil of moralistic language.

Others question the depth of the lab’s commitment to AI Safety, or at least believe that the decision to throttle the model’s cybersecurity, biological, and chemical capabilities will cause more harm than good, inadvertently or advertently making the lab the ultimate gatekeeper of competitiveness. If the company’s public product is quantitatively inferior in, say, neuro-scientific ability compared to that provided to chosen partners, then the lab will soon be able to, at least to some degree, choose winners and losers in an industry. A biotech company with access to a stronger model is likely to come up with more innovations, run quicker tests, and generally be more competitive. And, following this argument to its logical conclusion, it is not hard to imagine that the largest, most established players are more likely to be considered trustworthy than their smaller competitors.

This is, of course, only just getting to the most recent incident in this chain of events, in which on June 12th the U.S. government leveraged an export control directive to force Anthropic to suspend all access to the model ‘by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees’. In order to comply with the demand, the lab decided to suspend public access to the model entirely, making it the first time in history that a government forced an AI lab to rescind a public AI model. What was the reason for the directive? Well, the entirety of the truth has not yet been shared, but one potential spark was the discovery of a jailbreak method used to unlock some of the model’s cybersecurity capabilities, leading to government alarm. Regardless, Fable 5 remains blocked.

While the export control directive was likely nothing more than an attempt to force Anthropic to pull back Fable 5, the willingness to use language targeting foreign nationals’ access to a frontier model rightfully sparked international concern. One European analyst argued the severity of the directive showed plainly there is ‘no immediately effective and legible reason for [the U.S. government] or American developers to consider [international] interests in making access decisions’. In short, the world has no leverage in the AI ecosystem, and, without leverage, it has no agency in decisions around AI—how it is built or how it is deployed.

Of course, the best leverage for a region or state to reclaim sovereignty in this field would be to build its own frontier AI model. The issue is that this desire falls into what I call the Paradox of Advantage. The smarter and more capable AI models become, the more governments will see losing access as a threat to their security, thus deeming it necessary to build domestic champions. Yet, the smarter and more capable the frontier model becomes, the more consumers and enterprises will adopt that model, using it in countless every day interactions. As a principal bottleneck in AI model development is data, a model becoming the preferred provider creates a resource advantage that cascades into a compounding capability advantage. This advantage further threatens foreign states and attracts new users, on and on. The Paradox of Advantage is the return of the platform economy, but this time with greater implications and the same lacklustre solutions.

The events of last week showed the accelerating relevance of Artificial Intelligence, a technology with revolutionary potential. It is thus equally important to understand the principal tensions that drive its development and deployment. AI is a technology capable of inflicting serious harm, compounding entrenched economic interests, and becoming a cudgel aimed at political disagreements. It can also radically improve the world, expand economic opportunities, and democratise public capabilities. Here, we must be clear-eyed about the full range of possibilities—the prevailing narratives, all equally valid at the same time. An action to prevent misalignment in the artificial world can just as easily propel misalignment in the traditional world.

If, this time, the revolution will be technological, understanding the reasoning behind critical decisions made by leading actors, such as how to build a model and who can access it, will be just as important as the decisions themselves.