Tools for the Middle of the Work

What happens if you hand an AI agent a few thousand documents and ask it to find a story? Twenty minutes later, you get a confident report with citations. But aside from that artifact, you don’t get any way to know what the agent actually did. Did it scour the whole collection, or skim a tenth of it? Did it skip the most important folder? The report can’t tell you, and neither can the agent, really. You’re left with two bad options: trust it, or redo the work yourself.

In a prior piece, I argued against the empty prompt box - against compressing our work into one input and one output, and surrendering our judgment about everything in between. This piece is about what to build instead. Because if we want structure back in the middle of a task, we’re going to need lots of new tools.

Tools extend our capabilities in increasingly flexible ways. But they also give us rigid structures that enforce process. They provide constraints, narrowing the space of what’s possible to the task at hand. They provide visibility into what’s happening. And they provide steering, the ability to redirect work while it’s in progress. These handles on the work encode the tradeoffs we want to make for a task at hand: flexibility versus rigid process, autonomy versus hands-on verification.

My theory is that the current rush toward autonomous agents isn’t evidence that we want to hand off our work. It’s evidence that collaborating closely with AI is too expensive and requires too much cognitive overhead. That’s a tooling problem, and tooling problems are fixable.

I don’t think we need large, complex systems, and I don’t think we need to cut off the capabilities of advanced agents. I’m imagining a tiny tools approach: focused, composable, interoperable pieces of software we reach for as needed. Here are five, traced through a single hypothetical project.

A records request comes back, and you have 8,000 pages of documents. Agents should be really helpful here, but this is also where visibility becomes an issue. I’ve spent much of the past year building agentic systems for investigating document collections, and the most persistent friction point is that citing sources is not the same as showing coverage. More than knowing where a claim came from, a serious investigation needs to know whether the agent was thorough, whether it forgot something critical, or whether it padded its context with irrelevant material.

You can curate sources by hand, but that’s an ad hoc process of adding and removing documents by feel in response to the agent’s behavior. NotebookLM, for example, lets you toggle which sources are available for a query - and that’s about the extent of the state of the art.

A real tool here would show you a structured view of the collection and let you watch the agent move through it, telling you not just which documents it visited, but which regions it actually pulled into context. Then it would let you steer. You flag three documents as critical; the agent’s next pass shows you it read all three, plus a dozen adjacent ones you hadn’t noticed. You eject a folder of boilerplate that’s been polluting its answers. Instead of guessing at what the agent read, or manually auditing a massive autonomously generated report after the fact, you observe and direct the exploration to whatever level of rigor the story demands.

The documents seem promising for a story, so now you have a project: a reporting thread, data analysis, a draft to write, all on deadline. Linear, text-based planning - the kind agents are good at generating - works fine for a contained feature in a codebase. It collapses under anything more complex. A result comes back weird and reshuffles your priorities. The literature sends you sideways. Different pieces of the work need different levels of rigor, and the dependencies among them shift constantly.

Prose is the wrong format for that. A plan written in prose is a description of work; a plan you can manipulate is an instrument for doing it. The plan needs to be a durable, interactive object, a map of tasks and dependencies you can drag, collapse, annotate, and re-scope, where changing one node visibly propagates to the others. Project management software is starting to build integrations for delegating scoped work to agents, but what I’m describing sits one step earlier: the tool for sketching the broad shape of a project in diagrams and notes, before anything is delegated at all.

With a plan in hand, you start writing, and immediately run into the problem of levels. Writing happens at high altitude (brainstorming, research, outlining), middle altitude (structure), and low altitude (prose), and the process is not a descent from one to the next. Cognitive research on writing has made this point for over forty years: Flower and Hayes’ classic model describes composition as recursive movement among planning, translating, and reviewing. Real work bounces.

But agents don’t bounce; they cascade. Ask one for feedback on your outline and it starts rewriting your sentences. Ask it to find sources and it hands you a finished report. The agent has no concept of what altitude you’re working at, so it defaults to working at every altitude at once. You can improvise defenses: splitting levels across separate chat threads, writing increasingly baroque prompts. But that means holding the actual state of the project entirely in your head.

The tool we’re missing handles constraints across the hierarchy. Let me define the levels of my project, set expectations for each (e.g., at the outline level, you may reorganize but not draft; at the prose level, you may flag but not rewrite), and dictate how they relate. Then let me toggle among them explicitly, and hold the agent to the level I’ve selected.

Somewhere in the middle of the story is a critical number, a percentage that carries the central claim of the whole piece. It depends on an analysis pipeline the agent wrote and ran. Autonomous code generation is great for low-stakes tasks, but when the work is mission-critical, are you sure it ran the right pipeline? Are you sure the pipeline ran the right steps, in the right order, on the right data? Can you verify any of that without redoing the analysis yourself?

We need deterministic frameworks that make routine step-by-step analysis legible to a human. Orchestration tools exist (Airflow, Prefect, Dagster) but in my experience, agents struggle with them, and their framework overhead is wildly out of proportion to a one-off analysis. What I want is closer to a receipt than a framework: a lightweight manifest the agent emits as it works, verifiable at a glance against the code the agent claims it executed. Just enough structure to know for sure where a number came from.

Finally, the draft. A common theme in AI-assisted writing is wanting the agent in some parts of the process and locked out of others - help with grammar and structure, say, but hands off the prose. Right now those boundaries are nearly impossible to enforce. Agents emit full-file rewrites, and you have to hunt for what changed. Or you prompt for specific edits and then shuttle between a chat window and a document editor, doing the cognitive work of both reconciling the two and actually reviewing the substance. Tools like Codex give you file diffs, but diffs don’t work for prose. At best, they point you to the paragraph where something happened.

The funny thing is that we solved this problem for human collaborators decades ago: track changes and comments. There are early efforts at adapting that model to agents - Every’s Proof editor is one - but the integration isn’t there yet. The goal is an editing surface where the writer holds full control over which kinds of intervention are on the table, accepts and rejects at the level of the individual change, and never feels pressure to entirely delegate the piece itself.

Why don’t tools like these exist yet? Partially, it’s because we’re deep in an autonomous-agent hype cycle. Watching an agent one-shot an entire application makes for a great demo, and it aligns neatly with the incentives of organizations that want to hand routine work off wholesale. Intense collaborative processes are less flashy and harder to build.

But the focus on autonomy is backwards from what intensive knowledge work requires. We’re reaching for autonomous agents not because we want to surrender the middle of our work, but because the cognitive and logistical overhead of close, structured collaboration is too high with today’s tools. Every tool above is an attempt to lower that overhead, to make constraints, visibility, and steering cheap enough that we stop needing to choose between doing everything ourselves and handing everything off. Close that gap, and I think the demand for full autonomy starts to look like what it always was: a workaround. These five small ideas are a bet on a larger shift, one that lets us dig into hard problems alongside AI, instead of waiting outside the prompt box for an answer.