RSS Amplifier

AI Customer Research · Apr 30, 2026

Agents are now your primary readers

0
Sign in to vote or save

Caitlin Sullivan · AI Customer Research

✌️ Hey, I’m Caitlin. I help product, design, and insights folks do better customer research with AI—without the hype.

Dive deeper: Claude Code for Customer Insights (Sold out 2x! New dates coming soon) | AI Analysis Course (June enrolling) | more coming soon

Last week, multiple product and design teams kicking off agentic AI POCs asked me how they should be thinking about this:

“If quality of what goes into the AI workflow is going to determine the quality of what comes out — not just for humans — how should we set up documentation (especially of insights) so Agents can use it successfully?”

It’s a question I’m seeing more and more. AI agents are being deployed across the whole product delivery process, not just engineering or user need synthesis — discovery into findings, findings into PRDs, PRDs into design briefs, briefs into specs, specs into code for new or existing products. A feature factory where AI does the heavy lifting between discovery and working software.

Humans are shifting to “on the loop” rather than in it — setting direction, reviewing outputs, refining the process. Not doing every handoff by hand.

These teams are hinting at what everyone running a pilot like this bumps into: Documentation habits that have worked well enough for humans don’t work well enough for agents.

And AI is becoming your primary consumer.

The problem: What you write is now read by agents, and they take it literally. And when you don’t fill in the information gaps, they’ll fill them in — badly — on their own.

This edition walks through the key things that teams running agentic POCs need to consider:

  1. What good documentation looks like when AI is the primary consumer

  2. How to structure findings so they flow usably across research, product, and design

  3. Validation checkpoints that keep AI outputs grounded in real user insight

Plus a one-week pilot you can run immediately, checklist included.

  • 📍 New from me — workshop at Circus in London, new course is rolling

  • 🏕️ Base Camp — AI-ready docs from the ground up: the unit, the chain, the mechanics

  • 🗺️ The Route — what to set up for your POC this week: Monday moves, validation, one-week pilot

  • 🌰 Trail mix + 🌄 The view from here — extra reading and a model leaked last month

Let’s get into it —

  • Running a workshop in London: Claude Code for insights at experimentation conference Circus. May 11th. → Join me in person!

  • I launched a Claude Code for Customer Insights course this year. Cohort 2 is running right now. → Get in on the next round

🏕️ BASE CAMP

The fundamentals before any pilot: the atomic unit of an AI-ready doc, how those units chain across roles, and how agents actually read the chain when they get hold of it.

Whatever your team produces — a research finding, a PRD requirement, a design rationale, an ADR — the atomic unit of the findings they’re based on can often be the same: one claim, wrapped in six fields that an AI can actually work with.

One capsule per claim. One file (markdown, Notion page, Google Doc — whatever your team already uses).

These six fields are what travel with every claim through every layer. They’re non-negotiable for agents doing real work, and they’re exactly what also helps a human spot-check faster.

Most team artifacts cover the first two or three on the list. Rationale, constraints, and verification are where agentic work often breaks.

The structure stays the same across artifact types — you adapt the content to fit each.

Here’s a typical finding, then the agent-ready version of the same insight.

Typical finding:

Mobile iOS checkout: shipping cost is a major friction point

5 of 15 mobile iOS shoppers (33%) flagged shipping cost as frustrating during checkout — unprompted, across multiple sessions.

  • “I get to the end and suddenly there’s $12 of shipping. It feels like a bait and switch.” — P3

  • “By the time I see the total, I’ve already typed in everything. I just close the tab.” — P9

Recommendation: Reveal shipping cost earlier in the flow.

This is often solid enough for a stakeholder slide — a human gets the picture in ten seconds. But for an agent picking it up to run downstream workflows and decisions, source pointers aren’t atomic (no transcript timestamps to retrace), pattern strength and causal confidence collapse into one claim, no constraints scope it (mobile only? US only? new customers only?), no alternatives ruled out, no counter-evidence in view, no verification protocol. The agent confidently prescribes a full redesign — and a human on the loop has no fast way to challenge it without re-reading raw transcripts.

〰️

Here’s the same finding, with agent-ready documentation:

That’s one atomic finding. Scale it up: a PRD section is a handful of these stitched together; a design brief might be two of those capsules. The fields and structure stay constant. But one version above leaves agents to fill in gaps, and one doesn’t.

Humans are covered here, too. The capsule isn’t extra work — it’s the source everything else gets pulled from. Slack summaries can use Headline + a few key lines from other sections. Weekly deck slides pull Headline + 2–3 Rationale bullets, from all atomic findings. The PRD writeup embeds or links to multiple capsules verbatim. Execs get the Headline. Nothing has to be rewritten — everything is pulled and trimmed from the one file.

〰️

A finding doesn’t stop at the person doing the research. It becomes the source for a PM’s requirement, which becomes the source for a designer’s rationale. Each downstream artifact is its own capsule, citing the one upstream.

The same claim, used across three artifacts. Any reviewer — human or agent — can trace the brief back to P3 §2:40.

Break a link and the chain fails. A PRD that drops “causal driver: moderate” gets an agent confidently prescribing full redesign. A brief that doesn’t cite the PRD produces an agent optimizing for the wrong variable.

〰️

The feature-factory premise relies on agents walking this chain reliably. It’s worth being precise about what that actually looks like under the hood.

  • Memory vs. docs. The agent holds only its current context window. The chain lives in the docs — external, persistent, stateless to the model.

  • Re-reads. Every run. Each query triggers fresh retrieval via RAG or direct file access. No “last session” carries over unless you explicitly pass it.

  • Raw sources. The agent can pull them — but only if they’re indexed and linked alongside the capsules. “P3 §2:40” is a dead pointer unless P3’s transcript is in the retrieval system too.

  • Traversal. One hop at a time. Brief → agent retrieves PRD → which cites finding → agent retrieves finding → which cites transcript → agent retrieves quote. Structure enables walking backward; without explicit citations at each layer, the agent can’t.

A perfectly-structured capsule is still a dead pointer if the chain it sits in isn’t navigable.

🗺️ THE ROUTE

What to actually do — Monday morning through end of week one. Three setup moves, two checklists, one runnable pilot.

These are prerequisites. Skip them and the rest of the week unravels.

  • Pick one retrieval scope to start. Hard rule: the agent must reach both ends of every citation. The simplest way to guarantee that in week 1 is one scope — one Notion workspace, one repo, one Drive.

    • Monday: name the scope, audit which transcripts and briefs already live in it, and flag anything stuck in slide decks, locked PDFs, or email threads.

    • Multiple connected sources (capsule in Notion citing a transcript in Granola) work too, as long as each citation resolves to something the agent can act on, not inline text it just reads. That’s a week-3 problem; start simple.

  • Lock a citation format on day one. Stable IDs (P3 §2:40 — meaning participant 3, timestamp 2:40) need to survive renames, folder moves, and product-area aggregations.

    • Monday: pick the format, document it once, apply it to the next capsule. Don’t backfill old work — that’s a multi-day chore that doesn’t unblock the POC.

  • Replace vague references with real links. “See the research deck” is a phrase; an agent can’t follow it. Citations need to be links — to the capsule, the transcript, the prior finding.

    • Monday: take the most recent PRD or brief in flight, walk every reference to a research source, and fix any that aren’t actual links before the agent touches that doc.

〰️

Before a capsule enters any AI workflow, run these. Three or more “no’s” and the doc isn’t ready — fix it before trusting any output built on it.

☑ Headline states the claim in one sentence an agent can act on

☑ Source is specific, dated, addressable (P3 §2:40 — not “the interviews”)

☑ Confidence is labeled with counts and strength, not flattened into a recommendation

☑ Rationale names alternatives considered and what ruled them in or out

☑ Constraints are hard limits, not soft preferences

☑ Counter-evidence sits inside the capsule, not in a separate file

☑ Verification matches the artifact type — source-check / success criteria / acceptance criteria

☑ Raw sources (transcripts, source data) are indexed alongside the capsules so inline refs resolve

〰️

Three checks that work for humans on the loop.

  • Spot-check provenance. Sample AI outputs at random. Walk each citation back to a specific participant, quote, or source doc. If you can’t trace it → AI drifted. One to two minutes per sample beats re-reading a thirty-page output.

  • Contradiction sweep. Before accepting an output, ask the AI to find evidence against its own conclusion. If it can’t find any, be suspicious. Confirmation bias hits models as hard as humans — cheap to mitigate in the workflow.

  • Red-team with a skeptic. One person’s standing job: ask “where’s this coming from?” for every AI-generated claim. Not the most senior reviewer — the one whose only job is trust.

Why it matters. Anthropic’s framing of humans-on-the-loop oversight names three requirements: timely context, intervention authority, defensible rationale. These checks give you all three — context from provenance, intervention from red-teaming, rationale from contradiction sweeps.

📍 Save this section somewhere you can refer to during your POC.

〰️

🥾 TEST RUN

Don’t overhaul everything. For a four-week POC — or any team easing into this — I recommend running a one-week pilot on the highest-stakes artifact you have.

Before Day 1 — Pre-flight (Monday morning, ~1 hour). Run the three setup moves from the section above: name your retrieval scope, lock the citation format, fix vague references in the most recent in-flight doc. These are prerequisites for the capsule to function as an AI input.

Day 1 — Write one capsule. Pick a finding, PRD requirement, or design rationale your team is about to produce. Write it in the six fields. Use the worked example above as your template.

Day 2 — Render for humans. Pull the Slack summary, the deck slide, the PRD section. Note: nothing got rewritten — everything was trimmed from the one file.

Day 3 — Feed it to the AI workflow. Let the agent consume it as an input. Capture what it produces.

Day 4 — Run the post-agent checks. Provenance spot-check, contradiction sweep, red-team question. Note where the output held and where it drifted.

Day 5 — Debrief. Where the trust chain held → scale. Where it broke → the failure points tell you exactly what to fix before you let agents run more of the process.

Test this: in one week, with one capsule. If the structure holds, copy the template and make it the default for the artifact type. If it breaks, the miss tells you exactly which field needs more rigor before scaling the POC.

Two more reads on agentic work:

Last month’s accidental data leak revealed Claude Mythos — a new tier sitting above Opus, described internally as a “step change” and aimed squarely at enterprise customers.

The leaked drafts describe it as designed for “deep connective tissue between ideas and knowledge” — a very specific phrase for a model built to synthesize across documents, domains, and whole research corpora rather than answer one question at a time. Too expensive and risky for general release yet; going to cyber defenders first.

Why it matters:

As synthesis-focused models get majorly powerful, the quality of the inputs you feed them stops being a minor detail — poorly-structured findings and PRDs will produce increasingly confident-sounding wrong answers at a scale this tier is built for. And that will increasingly be on us, not “them”.

Next edition’s already drafted, shipping a bit sooner than usual — a small test based on feedback I’ve been collecting from many of you.

Keep moving,

— Caitlin

No posts

Read the original on aicustomerresearch.substack.com

Comments

Nothing yet. Say the first thing.

    Sign in to join the conversation.