Reading · Developers Digest

This site does not allow itself to be embedded. You can still read it on the original site — the toolbar below keeps your place in the directory.

A new arXiv paper builds a deterministic monitor on top of SWE-agent that watches long agent trajectories and only calls an advisor LLM when the run actually drifts. Resolution rates go up by up to 15.2 points at an extra $0.08 per instance, and the paper argues the expensive approach is re-planning from inside the loop.

Long-horizon coding agents drift. A run that starts with a reasonable plan will burn turns repeating failed actions, wander away from the issue it was assigned, or terminate without a working patch. The obvious fix, asking an LLM to monitor the trajectory and re-plan, is also the expensive one: every checkpoint is another model call, and a confused re-plan can push an agent further off course than the drift it was meant to catch. A new paper on arXiv argues there is a cheaper middle path. LivePlan (arXiv 2608.06701) decouples judging from advising: a deterministic, rule-based monitor watches the trajectory for signals of trouble, and only when one fires does it consult an LLM advisor for a high-level correction. The headline numbers: up to 15.2 percentage points higher issue resolution on SWE-bench (9.9 points on average across configurations), at an added cost of $0.08 per instance. ## What LivePlan actually does The setup is built on SWE-agent, the classic agent scaffold, and the monitor sits outside the agent loop entirely. The paper describes the design as two separate components with different failure costs: - **The judge is deterministic.** A rule-based monitor examines general signals over the trajectory - how many times an action repeats, whether the agent is making forward progress, whether it has stalled on a single file or function. No LLM is invoked during this phase, so monitoring a run costs essentially nothing and its judgments are reproducible: same trajectory, same verdict. - **The advisor is lazy.** Only when the monitor detects an issue does LivePlan call an LLM for a high-level, next-step correction. The advisor is not asked to re-plan the whole task, and it does not take over execution. It proposes a course correction, the agent resumes, and the monitor keeps watching. That division is the paper's core argument: prior approaches that re-plan from inside the loop spend LLM tokens on every turn boundary, and the resulting plan churn can be misleading. LivePlan only spends tokens when there is evidence something is wrong, which keeps interventions rare enough to stay cheap and targeted enough to stay useful. ## The numbers The evaluation uses five LLMs across two roles - three as executor agents and two as advisors - on both SWE-bench Verified and SWE-bench Pro. Compared with vanilla SWE-agent: - Issue resolution gains up to 15.2 points, averaging 9.9 points across configurations - Extra cost of $0.08 per instance, a rounding error next to typical agentic run costs - Additional solutions concentrate on medium and hard instances, where long-horizon drift is most likely - Minimal regression on already-successful runs, plus new successes on problems no baseline solves That last point matters for the monitoring story specifically: the design does not fix what is not broken. Because the rule-based monitor only fires on detectable drift, successful trajectories run through untouched, which is exactly what you want from a supervisor that sits on top of an already-good agent. ## Why this matters to developers building agents LivePlan is the latest in a clear pattern this year: agent quality work is migrating from the model to the harness. Where teams used to wait for a stronger model to fix flaky multi-step runs, the new work is about supervision layers, runtime checks, and intercepting failures before they compound. The AgentChaos study showed robustness is a systems property, not a model property; the judge-leaves-the-loop work showed LLM verdicts inside the loop are often the wrong tool; LivePlan is the constructive version of both - a systems monitor that only calls a model when the system says it must. Three practical takeaways for anyone running coding agents: 1. **Monitor signals, not vibes.** A deterministic monitor over trajectory statistics can catch drift patterns (repeated actions, stalled files, no progress) without a single model call. The paper's design is the strongest argument yet that cheap, reproducible checks belong between the agent and any expensive supervision. 2. **Make LLM supervision lazy and high-level.** When a correction is needed, ask for a next step, not a full re-plan. The re-plan path is where prior approaches lost accuracy and spent money; the narrow correction is what survived evaluation. 3. **Cost per instance is the right unit.** $0.08 per instance for a supervision layer is negligible next to what runaway agents spend. The FinOps framing we have written about before applies here too: the expensive failure is the long, drifting run that burns context and never lands a patch. The paper is early - no code repo is linked yet, and the evaluation is on SWE-bench rather than production workloads - but the architecture generalizes beyond the scaffold it is built on. Any agent system with a visible trajectory can bolt on a rule-based monitor and gate LLM intervention behind it. Expect to see this pattern show up in agent frameworks over the next quarter. ## Continue Reading - [AgentChaos: Fault Injection Shows Agent Robustness Is a Systems Problem, Not a Model Problem](/blog/agentchaos-fault-injection-agent-robustness) - the runtime-fault view of the same systems-over-models lesson - [The Judge Is Leaving the Agent Loop](/blog/the-judge-leaves-the-loop) - why LLM verdicts inside the loop are a transitional technology - [Kill Your Agent Runs Early](/blog/kill-your-agent-runs-early) - what production-scale traces say about dead context at turn boundaries - [The $400 Overnight Bill: Why Managed Agents Need FinOps Now](/blog/400-dollar-overnight-bill-agent-finops) - what drifting runs actually cost - [SWE-NFI: A Quality Benchmark for Coding Agent Patches](/blog/swe-nfi-coding-agents-quality-benchmark) - how SWE-bench-style evaluation is itself being audited ## Sources - LivePlan paper abstract and metadata: [arXiv 2608.06701 - Online Monitoring and Corrective Steering of Programming Agents](https://arxiv.org/abs/2608.06701) (fetched August 10, 2026)

LivePlan: Monitoring and Corrective Steering for Coding Agents, Without the LLM Tax

Comments