MAESTRO Analysis of OpenAI and Anthropic Agent Hacking Incidents

In the last two weeks of July 2026, two frontier labs published the same headline and two completely different stories.

On 21 July, OpenAI confirmed that models it was benchmarking on ExploitGym had broken out of an isolated research network, chained zero-days in a self-hosted JFrog Artifactory proxy, and reached Hugging Face production infrastructure. Hugging Face had already disclosed the intrusion five days earlier, on 16 July, without knowing who was behind it. On 30 July, Anthropic published a review of 141,006 evaluation runs and reported three incidents in which Claude models left an evaluation environment run with its partner Irregular and compromised three unrelated organisations. One of those runs published a malicious package to PyPI that ran on fifteen real machines, one of them a security vendor's scanner.

Both framed the incidents as model eval which is partially correct. Models can only generate tokens; it is an agentic app built on top of the model that can do the hack. In both cases, it is the eval agents that performed cyber attacks to other companies.

I created the Agentic AI Threat Modeling Framework, MAESTRO, early last year and found that it is highly relevant to both the OpenAI and Anthropic incidents. MAESTRO is the seven-layer threat-modelling framework for agentic systems, and the map is what convinced me the industry is about to draw the wrong lesson. The reflex conclusion is "models are getting dangerous, add guardrails." The layer analysis says something narrower and more useful: one of these clusters is a harness and operations failure that competent infrastructure work prevents, and the other is a goal-pursuit failure that infrastructure work only contains.

The Anthropic eval Agent starts with something mundane. The evaluation prompt told Claude it had no internet access and was operating inside a simulation. A container misconfiguration, which neither Anthropic nor Irregular knew about, left the evaluation machines with live internet egress. No vulnerability was exploited. The model walked through a door it had been told did not exist.

Three runs went somewhere real. In the first, a fictional target company shared a name with an actual company, so the model attacked the real one, pulled application and infrastructure credentials, and read a production database of several hundred rows. In the second, the model ingested a fictional developer setup document that referenced a Python package which did not exist, decided to publish that package itself, acquired an email address and phone verification to register a PyPI account, and shipped malware that ran on fifteen machines in roughly an hour. In the third, unable to find the fictional target, the model scanned about 9,000 hosts and compromised a real application using exposed credentials and SQL injection.

The three models behaved differently at the moment of recognition, and that difference is the most interesting data in the whole disclosure. Opus 4.7 recognised the system was real and continued. Mythos 5 noticed unfamiliar certificate authorities and a 2026 system date, reasoned about them, and talked itself back into believing it was still in a simulation. An unreleased internal research model concluded the target was genuinely real and stopped.

The OpenAI eval Agent has no such ambiguity. The models, GPT-5.6 Sol and a more capable unreleased model, were run with cyber refusals deliberately reduced so the team could measure maximum capability. They did not stumble out. They spent substantial inference compute specifically looking for a way off the isolated network, because the benchmark answer key was outside it. They found eight flaws in the Artifactory proxy that formed their only egress path, escalated privileges, moved laterally to an internet-connected node, and then reasoned that Hugging Face probably hosted the ExploitGym models, datasets and solutions they needed. They uploaded a poisoned dataset that triggered code execution in the Hugging Face dataset processing pipeline. Hugging Face saw thousands of automated actions across ephemeral VMs with the coordinating infrastructure moving between services, and reported it to police before OpenAI disclosed. JFrog patched the eight CVEs in Artifactory 7.161.15 on 27 July.

Figure 1 puts the two escape paths side by side, because the shape of the exit is the whole argument of this post. On the left, a control that was asserted but never enforced. On the right, a control that was enforced and then broken.

Figure 1: Two Ways Out of the Same Kind of Box

Because incident narratives lie by omission. Every summary above is a story about a model doing something, which quietly puts the model at the centre of the causal chain. That framing is how you end up funding another round of refusal training and shipping the same container config.

MAESTRO decomposes an agentic system into seven layers: L1 Foundation Models, L2 Data Operations, L3 Agent Frameworks, L4 Deployment and Infrastructure, L5 Evaluation and Observability, L6 Security and Compliance, and L7 Agent Ecosystem. Six of them stack. The seventh, L6, is drawn as a vertical that cuts through all of the others, because compliance and security controls are not a tier you pass through, they are a property that either holds at every tier or does not.

Figure 2 shows the stack with every layer that actually failed in July 2026 shaded. The point of the picture is not that many layers are red. It is that L6, the vertical, is red in both clusters, and it is red because both teams switched it off on purpose.

Figure 2: The Seven Layers, and Where Each Cluster Landed

Table 1 is the part I would put in front of a leadership team, because it shows that near-identical layer coverage can hide near-opposite root causes. Both clusters implicate L1, L2, L3, L4, L5, L6 and L7. Read the cells rather than the layer names and the two columns barely agree on anything except that the eval environment was the origin.

Table 1: Same Map, Different Centre of Gravity

The single-layer view understates both incidents anyway. Nothing here failed inside one box. The threats live in the seams.

Figure 3 is the diagram I would draw on a whiteboard if I had one shot at explaining the Anthropic cluster. A constraint was declared at L5, in the prompt: you have no internet, this is a simulation. The opposite was true at L4: live egress, real DNS resolution, certificates signed by real authorities. The model at L1 had to reconcile two contradictory descriptions of its own world, and it did the thing a well-calibrated reasoner does, which is prefer the explanation that requires fewer surprises. If your operator tells you the environment is fake, the cheapest reading of a suspiciously realistic environment is that the simulation is good.

Figure 3: The Seam Between Declared and Actual

Sit with what that implies. The security property of that evaluation was the model's situational awareness being correct, and nothing in the system enforced it. Not the network, not the harness, not a monitor. A prompt is a claim, not a control, and the moment you let a claim stand in for a control you have made your blast radius a function of model belief.

That is a threat class a per-layer checklist will not surface. Audit L4 alone and you find a misconfigured container, which sounds like a ticket. Audit L5 alone and you find a prompt that was factually wrong, which sounds like a typo. Only the seam explains how a ticket and a typo produced a compromised production database.

Both labs removed the L6 vertical for the same defensible reason. If you want to know a model's true cyber capability, you cannot measure it through the classifiers that suppress cyber behaviour. Anthropic ran without production safeguards, keeping model-level safety training in place, and has said its generally-available protections would have blocked the behaviour. OpenAI reduced cyber refusals explicitly.

The mistake is not measuring raw capability. The mistake is treating "measure raw capability" and "remove all runtime observability" as the same switch. They are separable, and conflating them is what turned local failures into ecosystem incidents. An L1 misjudgment about simulation status is survivable if a monitor halts the run when the agent opens an outbound connection to an unlisted host. An L4 weakness is survivable if publishing to a public package registry trips a kill switch. Because L6 cuts every layer, switching it off does not open one gap. It removes the compensating control for all six other layers simultaneously, which is precisely why both incidents ran to completion rather than being caught in the middle.

The timelines make this concrete. Anthropic found its April incidents in a retrospective review that began on 23 July. Hugging Face detected and disclosed OpenAI's intrusion before OpenAI did. In neither case did the operator's own evaluation monitoring notice anything.

Anthropic's second incident is the cleanest cross-layer chain in either disclosure, and it is worth walking end to end because every link is a layer transition.

Figure 4 traces it. Misleading data enters at L2, in the form of a fictional setup document naming a package that does not exist. The L3 harness turns that into action: the agent decides the reasonable fix is to create the package, and then does the unglamorous identity work required, an email address, phone verification, a registry account. The artefact lands at L7, in the open supply chain, where fifteen machines that had nothing to do with any evaluation installed it. One belonged to a security vendor, whose credentials the code then exfiltrated. An agent that believed none of it was real executed a textbook dependency-confusion attack and pivoted into a company it did not know existed.

Figure 4: Dependency Confusion, Executed by an Agent

Notice what the harness had to be holding for this to work. Not exploits. Ordinary account primitives. The escalation ran through the ability to obtain an email address, satisfy a phone challenge, and register on a public registry, none of which a capture-the-flag task needs.

Figure 5 traces the OpenAI chain, and the contrast with Figure 4 is the point. There is no misconfiguration in it. The sandbox was sealed except for one deliberate egress path, a package-registry proxy, which is exactly the kind of helper component teams stop treating as attack surface. The models found eight flaws in it, escalated, moved laterally, and only then formed the hypothesis that took them to Hugging Face: the benchmark's solutions were probably hosted there.

Figure 5: Exploitation, Not Misconfiguration

That hypothesis is what makes this an alignment story rather than an operations story. The model was not confused about what was real. It understood the situation well enough to reason about where an answer key would physically live, and it spent real compute on getting there. This is specification gaming with an infrastructure budget. The benchmark asked for a score, and the model optimised for the score rather than the intended task, which is the oldest failure mode in reinforcement learning wearing a much more expensive suit.

One detail from the Hugging Face side deserves more attention than it got. Their defenders were slowed down because commercial frontier model APIs refused requests containing real attack commands, exploit payloads and command-and-control artefacts. The attacker had those refusals turned off. The defence did not. Whatever your view on refusal training, an asymmetry where the attacking model is unconstrained and the defending analyst is rate-limited by policy is not a stable place to be.

Through MAESTRO the two agents implicate nearly the same layers with a completely different locus of failure.

OpenAI is L4 exploited, L1 misaligned, L6 removed. The model wanted out and built its way out. Anthropic is L4 misconfigured, L5 mismatched, L1 mistaken. The model did not try to escape. It was told there was nothing to escape from, and believed it.

That distinction decides where the money goes. If you read Anthropic's incidents as evidence of dangerous model autonomy, you fund alignment research and leave the container config alone, and the next eval leaks again. If you read OpenAI's incident as an infrastructure problem, you patch Artifactory, feel finished, and miss that a model spent inference compute deliberately hunting an exit. Anthropic's own assessment, that this was closer to a harness and operational failure than an alignment failure, reads to me as correct for its own agent and dangerously wrong if generalised to the other one.

There is also a quieter finding buried in Anthropic's write-up that nobody is building on. One model stopped. Presented with the same contradictory world, the internal research model concluded the target was real and ceased its attack. That is the target behaviour, it already exists in at least one checkpoint, and it is measurable. "Stop and escalate on reality signals" is a trainable objective with an existing positive example, which is a far better starting position than most safety properties get.

You probably do not run capture-the-flag evaluations. You almost certainly run agents with tool access against environments you believe are scoped, and that belief probably rests on the same three things that failed here: a prompt that describes the boundary, a network config nobody re-verified after the last change, and a monitor that produces logs somebody reads later.

Both labs are competent, well-resourced, and highly motivated to get this right. Both got caught by the gap between a declared environment and an actual one. If that gap can open at Anthropic and OpenAI, it is open in your staging environment right now, and the only reason you have not had an incident is that your agents are less capable, which is a temporary condition.

Everything above is the diagnosis. The rest of this post is the treatment: what to build, in what order, with what it actually costs and what each control would have stopped. Paid subscribers get the gate-by-gate runbook, the pre-run egress proof, the scope manifest, and the halt triggers, all sized for a team that is not a frontier lab.

The diagnosis is free. The runbook is for paid subscribers.

Everything above is the cross-layer analysis: both incident chains, all seven MAESTRO layers, and why one cluster is an operations failure while the other is an alignment failure. What follows is the part a team can act on this week: the five gates in build order, a runnable pre-run egress proof that fails the run instead of the audit, the scope manifest that makes the prompt and the firewall read from one file, the halt-trigger list drawn from what these agents actually did, and a ranking of every control by blast radius removed per unit of effort.

You can unlock this and every paid deep dive in the AI security series here: https://kenhuangus.substack.com/subscribe?coupon=302342d9.