“Regular” Code Is Still King Says Nvidia

On Friday, Nvidia published a result that should reorganize how you budget your next AI project. Its research team ran Claude Opus 5 against ARC-AGI-3, the interactive reasoning benchmark where an agent is dropped into unfamiliar 2D game environments with no instructions, no stated rules, and no stated goal, and has to work out what the buttons do by pressing them. ARC Prize reports Opus 5 at roughly 30% on the public set at high reasoning effort. That was the best score of any model tested.

Nvidia’s system, called Agentic Variation Operators, scored 100.00 RHAE. All 183 levels, all 25 environments, in 6,624 environment actions, roughly 12% fewer than the VISTA harness needed to clear the same levels driving the same model.

The model was identical. The difference was code.

Nvidia’s engineers wrote the conclusion themselves at the end of the post: “The model matters, but the model is not the entire agent.”

The word Nvidia uses is harness. Adel El Hallak, vice president of product in Nvidia’s AI unit, told TechCrunch that most of the industry treats an agent as though it were an API wrapper over a model. It is not. It is the model plus the scaffolding around it: the tools it can reach, the runtime it executes in, the skills and libraries you hand it, and above all the machinery that decides what it remembers.

AVO’s two load-bearing mechanisms are boring in the best way. The first is persistent memory that carries prior implementations, evaluation results, compiler and profiler output, and accumulated reasoning forward, so the agent resumes from state instead of rebuilding its search every time a context window rolls over. The second is a supervisor, a separate agent watching the broader trajectory and intervening when the main agent stalls, loops, or wanders back down a path it already exhausted.

Before ARC-AGI-3, Nvidia pointed the same architecture at GPU kernel optimization. AVO ran continuously for seven days, explored more than 500 optimization directions, and committed 40 kernel versions. The resulting multi-head attention kernels beat cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5% on DGX B200 systems. It then adapted the evolved kernel to grouped-query attention in about thirty minutes of additional autonomous work. Seven days of continuous inference against a single task, with nobody prescribing the steps.

Nvidia is careful about what the 30-to-100 comparison proves, and so am I. The company says explicitly that this was not a controlled ablation, that the ARC Prize baseline used a different reasoning setting and a different evaluation setup, and that the memory system’s individual contribution was not isolated. Fine. Nobody got to 100 by training a better model, and that is the part worth arguing about.

Nvidia is not the only lab that noticed. In July, OpenAI published a post-mortem on its own embarrassing ARC-AGI-3 numbers. GPT-5.6 Sol had scored 7.8%. GPT-5.5 scored 0.4%, which is roughly what you get from mashing buttons.

So OpenAI read the traces. The model was not stupid. The benchmark’s harness discarded the model’s private reasoning after every action, forcing it to re-derive the game from scratch on each turn, and it used rolling truncation, so as history grew the earliest actions silently fell out of context. The agent had been given amnesia and then graded on its ability to learn.

Turning on two settings OpenAI already ships in ChatGPT and Codex, retained reasoning and compaction, moved the score from 13.3% to 38.3% on their own reimplementation while cutting output tokens by six times. Their stated conclusion was that evals “rarely measure models in isolation.” Two configuration flags, three times the score, nothing retrained.

If that still sounds like a benchmark curiosity, Databricks priced it. In July the company published results from an internal coding benchmark built on its own merged pull requests against a multi-million-line codebase spanning Scala, Go, Rust, TypeScript, Java, Python, Bazel, and protobuf. These were tasks no model had trained on, graded by tests its own engineers wrote, with git history sealed off so agents could not walk forward and find the answer.

Their finding: “The harness a model is called from dramatically impacts cost and quality.” Running the same model at the same thinking effort through two different harnesses moved cost per task by more than 2x while quality stayed flat. The difference was context discipline. The cheaper harness sent roughly a third as much context per turn and finished in fewer runs.

The corollary breaks a habit most teams still have. Databricks found per-token price to be a poor predictor of what a task actually costs. Sonnet 5 is about 1.7x cheaper per token than Opus 4.8, yet on their tasks Sonnet cost $2.09 per task against Opus’s $1.94 and scored six points lower on completion, 81% against 87%, because it read more, worked longer, and burned 1.9x the tokens getting there. Ali Ghodsi’s summary to TechCrunch was that the wrong harness “can 2x your cost.” You are not buying intelligence by the token. You are buying finished work, and the harness sets the exchange rate.

The .NET world has been quietly ahead on this, which is not a sentence I get to write often.

Microsoft Agent Framework reached 1.0 GA on April 2, 2026, folding Semantic Kernel and AutoGen into one supported runtime with the same programming model across .NET and Python. At Build in June, Microsoft shipped something it named, without any apparent irony, the Agent Harness. Read the Nvidia paper first and the feature list reads like a checklist.

Automatic context compaction that monitors token usage and compacts chat history mid-loop, which is OpenAI’s ARC-AGI-3 fix as a framework default. FileMemoryProvider for session-scoped memory that survives turns, which is AVO’s persistent memory. AgentModeProvider, which separates plan from execute. BackgroundAgentsProvider, which fans subtasks out to child agents running in parallel. AgentSkillsProvider, which discovers and executes skills from the file system, turning El Hallak’s skills and libraries into something you register in DI. TodoProvider, so work items live in session state rather than in the model’s fading recollection. ToolApprovalAgent for human-in-the-loop gates on sensitive calls, OpenTelemetryAgent emitting semantic-convention traces, and a sandboxed ShellExecutor that is, notably, .NET only.

The whole thing turns on with one call, chatClient.AsHarnessAgent(...). Handoff orchestration is a few lines of AgentWorkflowBuilder declaring which agent may transfer control to which, with the framework injecting the handoff tools. Foundry Hosted Agents give you scale-to-zero with the filesystem intact on resume, per-session VM isolation, and OpenTelemetry flowing into Application Insights with no extra wiring.

Microsoft also shipped the honest version of the problem. Its CodeAct work opens by observing that “many agents aren’t bottlenecked by model quality” but by orchestration overhead, because every small tool call is another model round trip. CodeAct collapses that loop: the model writes one short program that calls your tools, runs it once in a Hyperlight micro-VM, and returns a consolidated result. On a representative multi-step workload, Microsoft measured 27.81 seconds and 6,890 tokens the traditional way against 13.23 seconds and 2,489 tokens with CodeAct. Half the wall clock, a third of the tokens, same model.

Underneath it, the memory story finally has a home that is not somebody else’s session store. SQL Server 2025 shipped a native VECTOR type and VECTOR_DISTANCE as GA, with DiskANN-backed CREATE VECTOR INDEX and VECTOR_SEARCH behind the preview flag. EF Core 10 exposes it as SqlVector<float> on your entity type, alongside full-text predicates and RRF hybrid ranking in the same query planner that serves your transactional data. Your agent’s long-term memory can live in the same database as your orders, under the same backup policy, the same row-level security, and the same DBA who already gets paged when it breaks.

I run exactly this shape at ShopSnap. Cara, the shopping assistant we roll across our merchants’ storefronts, does not fail because Claude had a bad day. She fails when the harness loses the thread. A shopper asks whether the jacket runs small, then two turns later asks her to compare two products and check out, and the routing decision made on turn one has not survived to turn two, so a conversion moment lands on the cheapest model in the stack. Or a tool call succeeds, nothing durable records that it succeeded, the model comes back around the loop, and the promotion gets applied twice. Every serious bug I have chased in that system has been a routing bug, a state bug, or an observability gap, and not one of them was fixed by changing models. That is why I wrote PLogger. You cannot debug a loop you cannot see, the model vendor’s dashboard is never going to show you your loop, and none of that is academic when the loop is running on a store that belongs to somebody else.

The benchmarks the industry used to sell model upgrades have gone flat at the top. Epoch AI’s April 2026 refresh put leading models clustered above 89% on MMLU-Pro, several inside a single percentage point of one another. GPQA Diamond sits near 94% and MATH-500 near 96%, both well past the point where they separate anybody. On SWE-bench Verified in March 2026 the top five were Claude Opus 4.5 at 80.9%, Opus 4.6 at 80.8%, Gemini 3.1 Pro at 80.6%, MiniMax M2.5 at 80.2%, and GPT-5.2 at 80.0%. Five frontier models from four labs, inside one point of each other. OpenAI has stopped reporting Verified scores entirely after its own audit found that every frontier model it tested could reproduce gold patches or problem specifics verbatim.

Open weights have caught up on the axis that actually pays. Databricks placed GLM 5.2 in its top capability tier, statistically tied with Opus 4.8 on quality at $1.28 per task against $1.94. MIT’s Initiative on the Digital Economy published work in January arguing for capability convergence: models on fixed training budgets close the gap on models with growing ones, because the leader hits diminishing returns first.

The economics point the same way. Frontier training runs already cost on the order of $500 million in compute, with the next generation projected between $1 billion and $10 billion. The curve everyone keeps buying into is a sigmoid that looks like a straight line only because it is plotted on a log axis. Plot it linearly and it is exponential decay, where every doubling of compute returns less than the doubling before it.

The models also still cannot be trusted with long work. In April, Microsoft researchers published DELEGATE-52, testing 19 LLMs on extended document-editing workflows across 52 professional domains, from coding to crystallography to music notation. Frontier models corrupted an average of 25% of document content by the end of long workflows, with degradation worsening as documents grew larger, interactions ran longer, or distractor files appeared. The finding that should stop you cold is that agentic tool use did not improve performance on the benchmark at all. The authors’ verdict was that current models are “unreliable delegates.”

And there is the thing nobody was modeling in 2023: the frontier is now partly gated by safety review rather than by loss curves. On August 7, OpenAI said it had suspended work on aspects of its upcoming Astra model after internal evaluation put it at or near a critical cybersecurity capability threshold. Whatever you make of the framing, the practical effect is that raw capability release has stopped being purely a function of how much compute a lab can buy.

I am not going to pretend the plateau case is airtight, because METR’s data cuts the other way and I would rather say so than have you find it yourself.

METR measures the length of task, in human expert hours, that an agent completes at 50% reliability. That horizon doubled roughly every seven months from 2019, then accelerated. Its Time Horizon 1.1 suite puts the doubling at about 129 days from 2023 onward and roughly 89 days from 2024. In a February to March 2026 pilot run with Anthropic, Google, Meta, and OpenAI, the strongest assessed agents landed near 16 to 20 hours on the 50% horizon. That is not a flat line. That is the steep part of a curve.

But read what METR says is driving it: greater reliability and better recovery from mistakes, combined with reasoning and tool use. Reliability and mistake recovery are precisely the properties a supervisor and a persistent memory layer supply. METR evaluates complete systems, not naked weights, which means some meaningful share of that curve is harness improvement wearing the model’s name tag. METR now also warns that estimates above 16 hours are unreliable because the suite itself is saturating, with only a handful of its 228 tasks running that long. When the people who built the ruler tell you it does not go that high, the number stops being evidence.

If capability is now mostly a systems problem, the answer is not a smarter model. It is more turns.

Every mechanism in this article buys accuracy with inference. The supervisor is a second model watching the first. Persistent memory means re-reading state you already generated. Compaction means paying a model to summarize context you already paid a model to produce. Fanning out to background agents means running several models where you used to run one. AVO’s seven-day kernel run, 500 explorations and 40 commits, is one task consuming a week of continuous GPU time.

The bill is already visible. Gartner forecasts worldwide AI-optimized infrastructure-as-a-service at $42 billion in 2026, nearly double 2025’s $21.5 billion, with $23.3 billion going to inference against $19 billion to training. That is the first year inference outspends training, and Gartner projects the market at $66 billion in 2027. The same analysis puts agentic workloads at five to thirty times the tokens of a standard chatbot interaction per task, and estimates of full multi-step, subagent-spawning workflows run three orders of magnitude above a single query.

Throughput confirms it faster than forecasts do. OpenRouter’s weekly token volume went from 5 trillion to 25 trillion in six months and is on pace to clear a quadrillion tokens across 2026. Fireworks AI was processing 10 trillion tokens a day in October 2025, 15 trillion by April, and 40 trillion by mid-July. Goldman Sachs Research models agentic AI multiplying consumer token consumption twelvefold by 2030. Morgan Stanley estimates that orchestration alone, the CPU work of coordinating all these loops rather than the GPU work of running them, adds $32.5 billion to $60 billion in data center CPU demand by 2030.

Per-token prices collapsed the entire time. Stanford’s AI Index tracked the cost of a GPT-3.5-class query falling from $20 per million tokens in late 2022 to roughly $0.07 by late 2024, better than 280x in two years, and enterprise AI bills went up anyway because volume outran price by more than price fell. That is what an agentic leap looks like from the accounting seat: unit costs in free fall and total spend compounding.

Nvidia, obviously, has an enormous financial interest in a world where a single task burns a week of GPU time. That is a reason to check the claim, not to wave it away. The checks exist and Nvidia did not run them. OpenAI found the effect in its own harness and published it. Databricks found it in its own repository and priced it. Microsoft has productized it in a framework you can read on GitHub this afternoon.

Stop shopping for models and start instrumenting loops. Measure cost per completed task, because per-token price will lie to you exactly the way Databricks showed it lies. Own your harness, meaning the compaction policy, the memory schema, the supervisor logic, and the tool approval gates, because that is where both your differentiation and your bill live, and a harness you rent is a harness you cannot debug at 2am. Put agent memory in SQL Server next to the data it reasons over so it inherits your security model instead of inventing a new one. Emit OpenTelemetry from every turn and keep the traces, because the failure you will have to explain to a customer happened forty turns ago inside a context window that no longer exists.

And budget for tokens the way you eventually learned to budget for cloud egress: not as a line item that shrinks as your engineering improves, but as one that grows as you succeed.

The model is the engine. You are still the one who has to build the car around it.

Gal Ratner is the founder and CTO of Inverted Software and WhiteStar Labs, and Chief Architect at Prana Entertainment in Las Vegas. He has spent close to thirty years shipping production software on the Microsoft and .NET stack for clients including Microsoft, Sony, Rockstar Games, 2K Games, Best Buy, and Allegiant Air, and was employee number six at Break.com. His current work is production agentic AI in C#: MCP servers, Microsoft Agent Framework harnesses, RAG pipelines on SQL Server 2025 vector search, and the PLogger observability framework. He is a Los Angeles Business Journal CTO of the Year finalist, a Brazilian jiu-jitsu practitioner, co-host of Edge Grip Podcast, and the author of the novel The Archive of Lost Suns. He writes about what actually ships.