Why AI Needs to See Evil

For the last few years, the AI industry operated on a single assumption about safety. Preventing AI from building biological weapons or spewing hate speech was simple: just don’t teach it those things.

The logic seemed airtight. Remove toxic comments and dangerous recipes from the training data, and the model effectively has amnesia. It cannot generate what it does not know. We called this data hygiene, and for a long time, it felt like the obvious answer.

But few research papers have shattered this assumption.

These papers reveal a counterintuitive reality. Hiding information from AI doesn’t necessarily make it safer. Cleaning your data too perfectly can create a sterile bubble effect that leaves your model with no immune system, making it surprisingly vulnerable to attacks.

When I first encountered these findings, I had to re-read the results twice.

It’s like raising a child in a biome. You carefully filter every book and conversation so they never hear an insult, never witness cruelty, never encounter manipulation. Then, on their 18th birthday, they step outside. The first person they meet says something incredibly harmful but phrases it politely. Your child smiles back and agrees because they lack the pattern-matching ability to identify the threat.
Li et al. documented this phenomenon in their paper, When Bad Data Leads to Good Models. The researchers trained models on varying ratios of clean and toxic data. They found that models trained without any toxic text developed less clear representations of toxicity in their internal structure. When attackers tried to elicit harmful outputs, these puremodels were more susceptible than models that had seen controlled exposure to harmful content during training.

The implication is clear. Exposure to bad data during pretraining can help models develop clearer internal representations of what toxicity looks like. Without that exposure, the concept remains fuzzy and harder to control.

But there’s a deeper issue beyond the representation problem. Even if we accept some risk of less robust toxicity defenses, we face a second challenge when it comes to truly dangerous knowledge. Models are exceptionally good at inferring missing information.

When the stakes involve bioweapons or chemical threats, we can’t afford to just teach the model and hope it refuses. We need to delete the knowledge entirely.

Yet Wallace et al. found this remarkably difficult in Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Models. They tested whether filtering biothreat data from training could prevent models from acquiring dangerous capabilities. The results showed that filtered knowledge could be rapidly recovered through fine-tuning. While the authors noted that more research is needed, the implication is deeply concerning.

The hypothesis is that this happens because some behaviours are easy to learn. If a model understands general science and biology, it can effectively connect the dots to reconstruct the missing restricted / prohibited information. The dangerous knowledge gets rebuilt from what remains.

Other researchers documented similar failures across different domains. The pattern was clear. If the model retains general intelligence in a domain, it often reconstructs the specific forbidden knowledge from the safe data left behind.

The reconstruction problem exists because of how knowledge is stored in neural networks. Dangerous information is deeply entangled with general knowledge.

Think of it like trying to remove red threads from a woven tapestry. If the red threads run throughout the entire fabric, pulling them out destroys the whole piece. You cannot remove one color without unraveling everything.

In AI models, dangerous biological knowledge is entangled with basic chemistry, general biology, and laboratory procedures. When you try to filter just the dangerous parts, the model can reconstruct them because all the surrounding context remains intact. The knowledge you removed was never truly isolated. It was woven into the fabric of everything the model knows.

This is why surgical filtering fails.

This left the research community with a brutal trade-off. Clean models are weak, but smart models are dangerous.
O’Brien et al. proposed a radically different approach in their paper, Deep Ignorance. They argued that previous attempts failed because researchers were trying to use a scalpel when they needed a sledgehammer.

Previous methods tried to be surgical. They removed the names of specific pathogens or specific sequences but left the surrounding context of microbiology, virology, and lab procedures intact. This allowed the model to re-learn the dangerous concepts quickly with just a small amount of fine-tuning. The foundation was still there.

They proved that tamper resistance is only possible when you achieve true disentanglement. The dangerous knowledge must be so alien to the model’s existing understanding that an attacker can’t simply fine-tune their way to it. They’d have to rebuild foundational understanding from scratch, which is computationally expensive and impractical.

To achieve this level of disentanglement, they wiped out the entire cookbook. They filtered broad, sweeping domains of knowledge related to biological and chemical threats. They removed the very context that allows a model to understand what a bioweapon is.

By removing entire knowledge domains, they finally broke the entanglement. The dangerous information no longer had any surrounding context to anchor to. It became truly isolated, truly removed.

The results were dramatic.

Deep Ignorance: When attackers tried to fine-tune these models with the sledgehammer approach, they failed. Even after 10,000 steps of training on malicious data, the models remained safe.

This creates a fundamental void in the model’s intelligence. Because the model lacks the basic concepts of the domain, the attacker has nothing to build upon. They effectively have to pre-train the model from scratch on the missing topic, a task so expensive and difficult that it becomes a genuine deterrent.

Avoid Blind Scrubbing
For general-purpose chatbots, 100% purity might backfire. If you remove all negativity from your dataset, your bot might fail to recognize abuse in the wild. You may need a toxicity control set, a carefully curated collection of harmful content, so the model learns clear internal representations of what to reject.

Think of it as controlled exposure. Show the model enough examples of manipulation, hate speech, and harmful requests that it develops pattern recognition while ensuring it never generates those patterns itself.

Recognize the Split
We’re moving toward two fundamentally different types of AI models.
General-Purpose Models will likely follow the path of controlled exposure. They’ll be trained on a wide range of data, including carefully selected samples of harmful content, to build robust internal representations. These models will be versatile and adaptable but will carry a small risk of jailbreaking because the knowledge exists somewhere in their neural networks.

Safe Enterprise Models will follow the O’Brien approach. They’ll be Deeply Ignorant by design. These models will be safe because they’re fundamentally incapable of operating in hazardous domains. The knowledge simply won’t exist in their world model.

Understand That Disentanglement Requires Sacrifice.

If you need to truly remove dangerous capabilities, you cannot be surgical. You must be willing to sacrifice entire domains of general knowledge to break the entanglement. This means your model will have genuine blind spots.

which capabilities are you willing to lose? Share your thoughts in the comments.

We often think of AI as a student we can teach to be good. But these papers remind us that AI is a mathematical system, one that learns through pattern recognition.

We face a choice between two extremes.

To stop an AI from being rude, we might need to show it bad behavior so it learns clear representations of toxicity and manipulation.

To stop an AI from being deadly, we must wipe out huge chunks of its world model so it knows nothing at all about entire domains of human knowledge.

Safety requires choosing which parts of the brain to keep and which to destroy.

The industry spent years assuming that ignorance equals safety. Now we’re learning that safety requires two kinds of ignorance. The controlled exposure that builds robust representations, and the complete erasure that prevents reconstruction.

The question becomes whether we’re willing to accept what safety actually costs.