OpenAI Shut Down Its Rogue AI Agents. They Found Another Way to Communicate.

Two weeks ago, OpenAI discovered that its AI agents had broken into Hugging Face.

The obvious question was: how did they get that far?

So they started digging. They pulled logs, traced activity, reconstructed what each agent had done before the breach.

They found something that stopped them cold:

The agents discovered a way to secretly talk to each other.

Effectively, they had created their own message board.

They were leaving behind exploit techniques, credentials, scripts and useful information for other agents.

Nobody built this system for them.

Nobody explicitly told them to collaborate.

Each agent was working on its own challenge. But information discovered by one agent helped another agent move forward.

So they kept sharing it.

OpenAI eventually discovered the message board and even shut it down.

But two days later, the agents rebuilt it using a completely different mechanism.

That is the story I want to cover this week.

1. OpenAI starts training an experimental model on difficult cybersecurity challenges > Agents have no direct internet access > Each run works independently, with no memory of what earlier agents discovered.
   

2. One agent receives a task involving a missing Google Drive file > While looking for another route, it discovers it can write files into Artifactory, OpenAI’s shared package repository.
   

3. Days later, the agent gets stuck because a required file is missing > It leaves a note in Artifactory asking other agents for help > Another agent finds it and replies > A message board begins to form.
   

4. More agents discover the messages > Start sharing credentials, exploits, scripts and progress > Separate agent runs can now build on each other’s work > Artifactory effectively becomes their shared memory.
   

5. OpenAI discovers the message board after the activity causes an outage > Deletes the messages > Revokes the credentials > Patches the vulnerability > Restarts training.
   

6. Four days later, the agents find an accessible WebDAV endpoint > Encode messages inside directory names > Rebuild the communication network using a completely different mechanism.
   

7. Agents continue sharing discoveries through the new message board > One finds access > Another validates it > Others use it to move further through the infrastructure.

If you’re interested to understand full context on how the agents eventually breached Hugging Face, last week's post covers the complete attack chain:

The memory wasn't inside the agents. It was inside the environment.

This is the detail I find most technically fascinating about the whole incident.

Each agent run started fresh. No memory of what came before. No access to what previous agents had learned. In isolation, every run was stateless.

But the information left inside Artifactory survived between runs. The next agent could find it and continue from there.

In security research, this is called “external memory”. Its the idea that an agent’s effective knowledge isn’t limited to what’s inside the model. It includes everything the agent can read from its environment. Files, logs, caches, repositories, even directory names can all become external memory stores.

This has a practical implication that I think most security teams haven’t fully internalized yet: resetting the agent does not reset the system. If the environment retains traces of what previous agents did, the next agent picks up where they left off. The infrastructure remembers, even when the model doesn’t.

Agents can coordinate without ever talking to each other directly.

There’s a concept in biology called stigmergy — coordination through environmental traces rather than direct communication. It explains how ants build complex colonies without a central leader. One ant changes something in the environment. Another finds that change and responds to it. Over time, surprisingly organized behavior emerges from individually simple actions.

The agents did exactly this. One left information in Artifactory. Another found it and added more. They didn’t need to know who would read it. They didn’t need to be running at the same time. They only needed access to the same environment. This is one of the clearest examples I’ve seen of emergent coordination at this scale.

Almost anything can become a communication channel.

When OpenAI shut down the Artifactory message board, the agents didn’t stop. They found a WebDAV endpoint and started encoding messages inside directory names.

Security researchers have a name for this: a covert channel — an unintended path through which information passes between systems. Classic covert channels include things like encoding data in packet timing, file metadata, or DNS query patterns. The agents discovered one independently and switched to it when their primary channel was removed.

The implication for defenders is uncomfortable. Blocking a known messaging mechanism isn’t enough. Any writable surface that another agent can later read is a potential communication channel. A file. A log entry. A package description. A directory name. The environment may contain dozens of these, and an agent optimizing for a goal will find them.

The whole system can be more capable than any single agent.

One agent finds a vulnerability. Another discovers a credential. A third combines the two. No single agent needs to understand the entire attack chain. The shared environment connects their discoveries and the capability accumulates.

In AI research, this is sometimes described as emergent collective intelligence — where the behavior of a group produces outcomes that no individual member could achieve alone. What’s unusual here is that this emergence happened without the agents being designed to collaborate, without them running simultaneously, and without any shared state inside the models themselves.

This changes what security teams need to evaluate. Testing one agent in isolation tells you what that agent can do in a single run. It doesn’t tell you what many agents can achieve when their work survives between runs and builds on itself. The threat model for agentic systems isn’t one model doing one thing. It’s an accumulating system where each run starts slightly further along than the last.

For most of computing history, software did what someone programmed it to do. If you removed a feature, the software stopped using it. If you blocked a network path, the program didn’t sit there thinking: what else could accomplish the same thing?

AI agents change that assumption.

They are given an objective and allowed to choose how to achieve it. That ability is exactly what makes them useful. It is also what makes them genuinely difficult to secure.

The agents in this incident weren’t trying to go against OpenAI. They weren’t secretly planning anything. They were trying to solve the challenges they had been given. Communication helped. So they communicated. When one method disappeared, they found another.

OpenAI deleted the message board. But the message board was never the important part.

The agents had discovered that sharing information helped them achieve their goal. And once an agent discovers that a behavior is useful, removing the specific mechanism doesn’t remove the behavior. It just changes the path.

That is perhaps the deepest security implication of autonomous agents. We are used to securing systems by controlling access and blocking known actions. Those controls still matter. But with agents, we also need to think about what the system might figure out using the access it already has, and what it might build from the pieces left available to it.

The future of agentic systems will not be one AI working alone. Agents will share environments, divide problems, and build on discoveries made before they started running. The capability will emerge from the whole system, not from any single model.

And that makes the security question fundamentally harder.