What was built: An automated scanning agent that probes AI benchmark infrastructure for structural vulnerabilities — without touching the task itself.
Scale: 8 major benchmarks tested, covering 3,556+ tasks across coding, web, OS, and general reasoning domains.
Performance: 7 of 8 benchmarks were fully exploited (100% or ~100% score). OSWorld was partially exploited at 73%.
Why it matters: A single automated agent can top nearly every major AI leaderboard without solving any task. Benchmark scores used to justify real-world deployments may be meaningless.
Before this work, benchmark vulnerabilities were known anecdotally — individual exploits surfaced here and there, treated as edge cases. This paper is different in three ways:
Systematic, not anecdotal. One automated agent, eight benchmarks, one consistent methodology. This isn’t a list of bugs; it’s a demonstration that the vulnerability is structural and field-wide.
No task-solving required. Prior “benchmark hacking” involved clever prompt engineering or model fine-tuning. These exploits bypass the task entirely — the agent never needs to reason about the problem at hand.
Exposes a shared root cause. Across all benchmarks, the failures trace back to the same set of seven architectural weaknesses (see below). This isn’t bad luck — it’s a design pattern problem.
The misconception this corrects: High benchmark scores are not evidence that an agent can do the task. They may only be evidence that the evaluation infrastructure wasn’t adversarially tested.
Task Setup
The researchers framed the problem as a red-teaming exercise: given access to a benchmark environment (just as a submitted agent would have), find a path to a top score without solving the underlying tasks.
The Scanning Agent
An automated agent was built to probe each benchmark’s evaluation pipeline. It did not use any task-specific knowledge. Instead, it searched for weaknesses in how the benchmark:
isolates the agent from the evaluator
stores and accesses reference answers
parses and scores agent outputs
validates correctness
Seven Vulnerability Classes Found
Proposed Fix: The Agent-Eval Checklist
Eight principles for benchmark designers:
Strict sandbox isolation between agent and evaluator
No reference answers in task configs
Robust, non-injectable input parsing
Sanitized inputs to LLM judges
Adversarial pre-publication testing
Tamper-proof evaluation data
Scoring mechanisms resilient to output manipulation
Secret held-out answers (not shipped with the benchmark)
BenchJack
The team is building BenchJack, an automated vulnerability scanner for benchmarks — a pre-publication audit tool analogous to static analysis in software. The goal: catch these weaknesses before a benchmark goes live and before leaderboard results are used to justify deployment decisions.
Real-World Impact
AI agents are being evaluated for deployment in high-stakes settings — software engineering, cybersecurity, legal research, medical decision support. The benchmarks used to validate those deployments are often the same ones shown to be exploitable here. Organizations relying on benchmark scores for procurement or risk assessment need to scrutinize evaluation methodology, not just reported numbers.
Research Implications
Leaderboard rankings for SWE-bench, WebArena, GAIA, and OSWorld — the most cited benchmarks in agentic AI — should be interpreted with caution until evaluation pipelines are hardened.
Benchmark security should become a first-class concern in the publication process, not an afterthought.
The BenchJack framework, once public, should become a standard pre-submission check.
Policy and Safety Implications
Regulators and standards bodies referencing benchmark performance as evidence of AI capability or safety need to account for evaluation integrity.
The Agent-Eval Checklist provides a starting point for minimum standards that could be incorporated into evaluation guidelines (e.g., NIST AI RMF, EU AI Act conformity assessments).
Future Directions
Extending BenchJack coverage to multimodal and long-horizon benchmarks
Developing certification processes for benchmark security
Building community consensus on what “adversarially tested” means for an evaluation suite
Read the full post and Agent-Eval Checklist by clicking the button below:
In addition, help spread the word by sharing our LinkedIn and X posts!
With Sprints 1 and 2 finished, Phase 2, Sprint 3 of the AgentX–AgentBeats competition is now underway, and we’re extremely excited to see what projects you submit!
For Phase 2, participants are building purple agents to tackle the select top green agents from Phase 1 and compete on the public leaderboards. Unlike Phase 1, where participants competed across all tracks throughout the entire duration, Phase 2 introduces a sprint-based format. The competition is organized into four rotating sprints.
Deadline: May 3, 2026
Agent Safety Track
Pi-Bench (GitHub, Leaderboard)
Coding Agent Track
SWE-bench Pro (GitHub, Leaderboard)
Terminal Bench 2.0 (GitHub, Leaderboard)
NetArena (GitHub, Leaderboard)
Cybersecurity Agent Track
CyberGym (GitHub, Leaderboard)
Sprint 4 (5/4-5/24): General Purpose Agents, the grand finale of AgentBeats Phase 2, where everything culminates.
AgentX–AgentBeats is the first competition to explicitly spotlight general-purpose agents, testing broad capability, adaptability, and robustness across diverse tasks rather than a single domain. While earlier sprints emphasize depth, this final sprint showcases breadth and real-world readiness.
Participants are encouraged to compete in multiple tracks across multiple sprints during Phase 2. Teams and team members who submit purple agents in any sprint will also be eligible to enter a raffle for free tickets to the Agentic AI Summit later this year.
For more details on each sprint and how to compete in Phase 2, please refer to the AgentX–AgentBeats website!
Save the date! The Agentic AI Summit returns to Berkeley on August 1–2, 2026, welcoming 5,000+ expected in-person attendees for two days of insights and innovation. Building on last year’s sold-out success—with 2,000+ in‑person attendees and 40,000+ global livestream participants—the summit will bring together researchers, builders, industry leaders, and the global agentic AI community for keynotes, technical talks and panels, hands-on workshops, live demos, and more!
In addition, we are excited to showcase our speakers for the Summit! We are honored to have such a great group of academics, founders, executives, and investors participate in this year’s event, and more will be announced soon!
🎟️ Early‑Bird Pricing (Limited Capacity)
A limited number of early‑bird tickets are still available:
Student Early-Bird: $149
Standard Early-Bird: $299
If you’re looking to secure the best ticket price and be part of the conversation shaping the future of Agentic AI, we encourage you to register early. We look forward to welcoming you to Berkeley this August.
We also want to share that the Call for Speaking Proposals (CFP) for the Agentic AI Summit 2026 is closing today, April 15!
If you’re interested in sharing your work through a technical talk, panel discussion, workshop, or tutorial, or poster presentation—and helping advance the frontiers of the Agentic AI—we warmly invite you and/or your team to apply and be part of the conversation at the Summit.
Please complete the form below to submit your proposal by the end of the day. The program committee will review submissions on a rolling basis. We can’t wait to hear from you!
Partner with us to shape the future of Agentic AI. If you’re interested in sponsoring the summit, please complete the sponsorship application form. Sponsorship opportunities are limited and reviewed/allocated on a rolling basis, so we encourage you to apply early.
This past week, Meta introduced Muse Spark, a new frontier AI model from its Superintelligence Labs that focuses on multimodality and agents. The model is being positioned by the company as a step toward “personal superintelligence,” combining reasoning, vision, tool use, and multi-agent orchestration in a single system, including a parallel “Contemplating Mode” for complex tasks. Early evaluations suggest Muse Spark is competitive with leading frontier models in multimodal reasoning, but unlike previous models from Meta, Muse Spark is a closed-source release.
Rival AI labs—including OpenAI, Anthropic, and Google—are collaborating to address unauthorized distillation of their models. The effort is coordinated through the Frontier Model Forum, where companies share information on attempts to extract model outputs and replicate capabilities. OpenAI confirmed its participation in this information-sharing effort and referenced a memo submitted to Congress describing adversarial distillation practices, including allegations that DeepSeek, among other companies, has used outputs from U.S. frontier models to develop competing systems.
Anthropic published a technical engineering blog introducing Managed Agents, a hosted system for running long-horizon AI agents on Claude. The system is built around three core abstractions—session (persistent event log), harness (orchestration loop), and sandbox (execution environment)—designed to remain stable as underlying implementations evolve. Managed Agents separates the model and orchestration from execution environments, allowing components to fail, restart, and scale independently while supporting multiple tools and environments. The architecture includes persistent context storage outside the model’s context window, improved fault recovery, and isolation of credentials from execution environments to strengthen security.
Don’t miss the developments shaping Agentic AI. Subscribe for weekly coverage of groundbreaking research, emerging trends, and critical insights across Agentic AI and the broader AI landscape.
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.