Slop, Speed, and Discipline: Hard Truths About AI Coding Agents

There’s a story I keep hearing in the ML community right now, and it goes something like this: a team spins up a fleet of coding agents, ships features at 10x speed, and celebrates. Two months later, nobody on the team understands the codebase anymore. Tests pass but production breaks. The agents are called in to fix the mess, but the codebase is now so large, so tangled, and so far beyond anyone’s context window that even the agents can’t get their bearings.

This isn’t a hypothetical. Mario Zechner, an independent software developer, who just shipped PI, his own coding agent harness, watched this pattern emerge and decided to say something about it out loud. His talk, “Building PI in a World of Slop,” is one of the most honest and technically grounded takes I’ve seen on the current state of AI coding agents. I agree with nearly everything he says. Here’s my take from his talk.

Mario didn’t build PI out of spite. He started using Claude Code in April 2025, loved it, and credits the Anthropic team as genuinely talented. But over time, something shifted. The tool grew faster than it improved.

His specific complaints are worth naming:

• Context he couldn’t control. Claude Code manages the context window on your behalf, and it does things behind your back. System prompts change on every release, including tool definitions. Tools get removed or modified without notice.

• System reminders injected at the worst moments. The tool would insert reminders into the context mid-task, sometimes with language like “this may or may not be relevant to what you’re doing”, which, as Mario notes, is exactly the kind of hedging that confuses a language model trying to maintain coherence.

• Zero observability. You can’t see what your agent is doing or why. For anyone who cares about understanding their systems, this is a serious problem.

• Shallow extensibility. Hooks exist, but they’re process-level, a new shell command spawned on each trigger. Not deep integration. Not programmable.

These aren’t complaints about Anthropic’s intentions. They’re complaints about what happens when a tool optimizes for feature velocity at the cost of transparency and stability. Mario’s analogy is perfect: if your hammer breaks every day on a construction site, you get mad. Developer tools are no different.

He looked at alternatives, shoutout to AMP and Factory Droid as the “Porsche and Lamborghini” of coding harnesses, and also dug into OpenCode’s internals, where he found similar issues: context pruning that effectively lobotomizes the model, LSP error injection mid-edit (checking for errors after every line, not after you finish your work), and a server that by default exposes itself to any website open in your browser.

So he built PI. And what he built teaches us something important.

PI ships with four tools. Its system prompt fits on a single slide. That’s the whole thing.

Before you dismiss this as toy-project minimalism, consider Terminal Bench, a coding agent benchmark that gives the model exactly one capability: send keystrokes to a tmux session and read the output. No file tools. No sub-agents. No search. Just a terminal. As of December 2025, Terminal Bench scores higher on the leaderboard than most full-featured harnesses, including native model harnesses. The simplest possible environment outperforms the most feature-rich ones.

This should give us pause.

We’ve been operating under an assumption that more context management, more tools, more agentic scaffolding equals better performance. Mario’s argument, backed by benchmark data, is that we’re in what he calls the “fuck around and find out” phase of coding agents. We don’t actually know what the right harness looks like yet. And in that uncertainty, the minimal approach wins because it introduces fewer confounding variables, gives the model cleaner signal, and is easier to reason about.

This maps directly to something ML practitioners already know: simpler models with strong inductive biases consistently outperform overparameterized ones in low-data regimes. The principle holds in agent design too. A model that’s been RLHF-trained extensively on coding tasks already knows what a coding agent is, you don’t need 10,000 tokens to remind it. It knows because it was trained to be one.

PI’s extensibility philosophy reinforces this: instead of shipping every feature, ship a minimal core and let the agent extend itself. Users describe what they need; PI builds the extension. The agent adapts to your workflow, not the other way around.

This is the part of Mario’s talk that I think every ML team lead and practitioner needs to hear.

Agents compound errors. Mario calls these “boooos”, mistakes that accumulate silently, without the friction that would normally alert a human developer. A human feels pain. They get confused, frustrated, they slow down, they refactor. Agents don’t feel pain. They will happily keep generating code into a broken codebase indefinitely.

The math here is brutal. With one human developer, you have a natural bottleneck on how many errors can enter the codebase per day. Add ten agents running in parallel, and that bottleneck disappears. The error rate scales. The review burden scales with it, but your human capacity to review does not.

And here’s the structural problem with what agents learn: they were trained on the internet, which is overwhelmingly our old, mediocre code. Not the pearls, the garbage. Every blank left in a spec gets filled in by the agent using patterns learned from that garbage. You get abstraction layers nobody asked for, duplicated logic, backwards-compatibility shims for scenarios that don’t exist. Enterprise-grade complexity generated in two weeks by two humans and ten agents. Congratulations.

“But we have a detailed spec.” Mario’s response to this is worth quoting directly: a sufficiently detailed spec is a program. If there are gaps in your spec, the model fills them. You don’t get to control what it fills them with.

“But we have a review agent.” Also not enough. Review agents catch some issues. They miss the architectural ones, the decisions that seem locally reasonable but cause global damage, because those require understanding the whole system, which is exactly what gets lost when no human is reading the code.

Mario’s framework for responsible agent use is practical and I endorse it:

• Scope tightly. Give the agent only the context it needs and guarantee it can find everything relevant to the task. If you can’t scope it, don’t delegate it.

• Modularize your codebase. Smaller, well-bounded modules make agentic tasks tractable and reviewable.

• Use agents for the right tasks. Reproduction cases for bugs, boring boilerplate, non-critical automation, research, these are good. Core business logic, security-sensitive code, architectural decisions, these are not.

• Read the critical lines yourself. Every one of them. If you don’t know what’s critical, that’s your answer: read more code.

The last point is uncomfortable, and I think that’s the point. The discipline to read your own code is not a tax on productivity, it’s what makes you the developer rather than the reviewer of whatever the agent decided.

Briefly, because it deserves mention: AI-generated GitHub issues and pull requests are burning out open source maintainers. Mario showed his own tracker, half of it is garbage from agent instances running autonomously, submitting PRs without reading the contribution guidelines, flooding issue queues with low-quality reports.

His response, auto-closing agent PRs, asking contributors to write issues in their own human voice under a screen’s worth of text, building a filter that only lets through accounts that actually read the instructions, is clever. But the fact that he has to spend time building abuse filters for his own open source project is a problem the community created.

If you’re using agents to contribute to OSS, please configure them with appropriate constraints. Or better yet, read the repo and write the issue yourself.

Mario ends his talk with something that sounds almost contrarian in 2025: slow down. Think about what you’re building and why. Learn to say no. Fewer features, but the ones that matter, and then use your agents to polish those features until they’re excellent.

I think he’s right. The race to maximize agent output is a race to maximize technical debt, user confusion, and maintainer burnout. The practitioners who will build the most durable, trustworthy AI-powered systems are the ones who treat agents as precision instruments, not autonomous co-developers.

Mario’s closing line is the one I keep coming back to:

“Friction is the thing that builds understanding of the system in your head, and it’s also where you learn new things.”

We’ve been treating friction as the enemy. It’s not. It’s the mechanism by which understanding transfers from code to developer. Remove it entirely and you end up with a codebase you can’t debug, can’t extend, and can’t trust.

Use your agents. Use them well. But keep your hands in the code.

Mario Zechner’s talk “Building PI in a World of Slop” was presented at AI Engineer conference. PI is open source, you can find it on GitHub. If you found this useful, forward it to a colleague who’s been talking about going “full agents.”