The Landscape
On April 7th, 2026, we published an article about two new kinds of vulnerabilities that were obvious ways to hack stochastic AIs (e.g., any language model). The focus of this article is on just one of them:
AI Takeover (AIT)
In the original article, we were speculating on the possibility of AITs. It focused principally on the Claude Code (CC) source leak as a vehicle to hack underlying AI systems. Our original hypothesis of how AITs could manifest — augmenting the model’s behavior directly — has, unfortunately, been validated (e.g., ClawHub malicious skills). But the dangers of AIT are significantly larger than even I realized in the first article. What is critically important to understand is that AI takeovers will likely impact everyone. You don’t need to have any involvement with AI, directly.
I have two primary goals in writing this article:
(1) To more clearly explain what AI takeover (AIT) is.
(2) To provide examples of AIT and the problem most people seem to be missing.
The principle danger with today’s language models (LMs) is in their lack of sound and exhaustive verification on the operations they perform. As a community, we must place validation and verification of AI inference as a first-order concern. If we don’t, these types of attacks / hacks will continue with likely increasing scope and damage. Due to the potentially unbounded LM behaviors, users and companies that give them the authority to perform operations will come with risk that is proportional to the power of the operations they are granted. AITs have moved away from being a hypothetical belief to a realizable exploit that has been successfully achieved multiple times. There are at least three AIT examples that I’m aware of:
Each of these exploits may appear to be disconnected, but they are all part of the AIT family. They only differ in the delivery mechanism that triggers the AIT event.
Most importantly, you cannot solve the underlying foundational problem with any band-aid “fix” that many of the companies are seemingly putting in place. Those quick fixes only resolve the exact route of the exploit that the AIT performed. They don’t address the fundamental problem: LMs do not have a semantics understanding of what they are doing.
What is AI Takeover?
An AI takeover (AIT) is when a stochastic AI system is hacked to do things that it was not intended to do. Unlike a traditional software hack, AIT doesn’t inject malicious code: it infects the AI/ML model itself. Once that happens, the model may appear to be functioning perfectly. You probably won’t even know the AI has compromised. If done properly, the prior “good” behavior will still exist. The AI will have the appearance of being a “good” actor. But secretly it will now also be capable of doing “very bad” actor things. What makes AIT notably more dangerous than traditional hacks is the variety of bad actor things it can do. These behaviors significantly exceed the bounded behavior of traditional software exploits.
AITs aren’t just one thing. It’s not a ransomware system. It’s not a phishing system. It’s not a data theft system. It could be all of them at once. Because it’s stochastically AI-driven and advanced AIs can be modified — residently (i.e., in-place) — to perform more types of operations, the surface area of AITs can evolve to become more malicious as the AI is enhanced with more malicious behaviors through prompts, skills or direct model augmentation.
AITs are Specific to Language Models (or Stochastic AI/ML)
AITs can only happen within the space of stochastic AIs (e.g., any type of language model, LLM, SLM, etc.). I want to try to clearly articulate this as there seems to be a general misunderstanding of what’s going on. This is not a general failure of AI/ML science and technology. Deterministic AIs cannot emit AITs. For an AIT event to trigger, one needs to (i) leverage a stochastic AIs and (ii) ensure that AI does not have a complete semantic understanding of actions it performs. If either of those two prerequisites don’t exist, there will be no AIT event.
With key advances in small language models, the possibility of these being locally resident and doing bad things in-place is increasingly more likely. There’s a lot more to unpack here, but we’ll save that for a deeper dives later. Right now, the important part is that we understand the breadth and the depth of AITs.
There appears to be a large amount of confusion thinking an AIT event is just another “bug” that can be casually fixed. There are no “bugs” to be fixed here. There is a fundamental architectural problem. AITs are a novel and generalized form of attack due to limitations of semantic reasoning. The only way to properly address AITs is for the underlying stochastic AI — itself — to be enhanced to include a complete semantic understanding of all of the operations it’s performing. Until then, we are just putting band-aids on broken arms. It’s just a kludge.
How to (Partially) Protect Yourself
I’m just beginning to understand the theoretical space and practical application of AITs. We only recently discovered their theoretical viability. Now we have evidence that our theories are not just theories, but realizable. Yet, none of the AI/ML, security, or software experts that I know — including me — understand them well enough to offer up novel solutions to them. Because of that, we can only try to address AITs is on previously known solutions to these types of problems. The classical approach is to build a full semantics reasoner into the LM that is both sound and exhaustive. As with most frontier science, soundness is relatively straightforward; exhaustiveness is hard. To the best of my knowledge, the state-of-the-art AI, formal, and software (and hardware) solutions all lead us to the same path: a computationally intractable solution. In simpler terms, AITs can’t be solved the right way with today’s technology.
I realize this isn’t going to be a popular take. I’ve tried to present the details of a novel type of exploit in AI, yet I have no clear and pragmatic solution. That’s not the way I wanted to end this article. Yet, as the head of GTT, it’s my first responsibility to share with all of you (i) what I think I know and (ii) what I think I don’t know. This unfortunately is one of those cases where I’m pretty sure I don’t know the answer (and the answer may not actually exist). Or, maybe it does, and I’m just not aware of it. Maybe one of you reading this know how to solve it and can share with me (and all of the rest of us) how it can be practically solved at its root? I’d love to hear that.
Using a fully sandboxed, air gapped VPS is a strong first step. But that only protects us — as users of AI. It doesn’t solve the problem where companies recklessly use stochastic AIs for automation tasks that they don’t semantically understand, which could negatively impact us and them (e.g., Meta’s AI Support AIT hacking IG accounts and Grok’s AIT costing them $200k). So long as these companies use LMs without comprehensive semantics reasoning, all of those people who use their service — including them — are at risk. Meta’s and xAI’s “fix” to their AIT, most certainly doesn’t include a full formal semantics reasoning system, because that’s computationally impossible with today’s technology.
Moreover, all the reports I’ve seen on these types of attacks seem to be missing this point entirely. They are focusing on resolving the known exploited AIT path (e.g., X user takes $200k from Grok, ClawHub’s malicious skills, Meta’s AI support system hacked). So they patch that one path. That’s one path of millions. That’s not the problem. The problem is in eliminating the possibility of AI takeover events.
Going back to the basics, one of the most interesting behaviors that nearly all human beings have that today’s AIs don’t is common sense. In both of the user-invoked AIT events, a human with even a marginal amount of common sense wouldn’t have made those mistakes. No reasonable human would just hand over $200k in crypto. They certainly wouldn’t give a random user full control over other people’s accounts.
As simple as it sounds: what today’s AIs need most is some common sense.
Goju
https://www.youtube.com/@gojutechtalk
https://discord.gg/HhcqqAXwk
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.