Reading · 🕹 prodmgmt.world | Becoming Top PMs Together

🕹 prodmgmt.world | Becoming Top PMs Together · 🕹 prodmgmt.world | Becoming Top PMs Together

Hello!

This is 🕹 prodmgmt.world | Becoming Top PMs Together

🆕 In today’s edition:

🆓 /edge-cases: a simple skill to catch missed edge cases

🆓 /working-backwards: the new PM skill is building smaller tests

🆓 PM OS 2.5 Is Out!

There’s a real PRD sitting in my examples folder, a Shopify to QuickBooks connector, and buried in feature one it says this, word for word:

“there could be 2 or more orders, whose aggregate sum should equal the payout amount. (if it doesn’t then something is wrong 😁)”

An emoji. Where the error handling should be. The PRD names its own failure condition, the money doesn’t add up, and then specifies nothing about what the system should do when it happens. Smiley face, next feature.

And honestly, I get it. A human engineer reads that line, snorts, and asks in refinement: “so what do we do when the sums don’t match?” The gap becomes a question, the question becomes an answer, and all of that happens before anything ships. That emoji was load-bearing for years and nobody noticed, because there was always a human on the other end to catch it.

An AI coding agent doesn’t snort. Feed Claude or Cursor that PRD and it won’t pause at the 😁. It picks something plausible, log a warning, book the invoices anyway, writes it with full confidence, and moves on to the next feature. The code compiles, because the tests you didn’t write can’t fail. Nobody finds out until the books don’t reconcile.

I think what I arrive at is this: the reason you write specs has changed, or at least the failure mode has. A human reader plus an incomplete spec produces a follow-up question, which is annoying but recoverable. An agent reader plus an incomplete spec ships wrong code anyway, and that’s much harder to catch.

The PM writing specs for an eng team had a backstop they probably never noticed they had: someone else’s confusion. The PM writing specs for an agent has that backstop removed. Nobody took it away on purpose, it just quietly isn’t there anymore.

The fix isn’t “think harder about edge cases” as a vague note to yourself before refinement. I’ve written that note. It doesn’t work. What does work, as far as I can tell, is a structured pass across eight categories, every time, in the same order: user types, contexts of use, unexpected inputs and system failures, user error, feature interactions, load, security and privacy, accessibility. The order matters because it catches the categories you personally always forget, and everyone has ones they always forget.

So I ran that pass on the connector PRD. The spec itself looks done: eight features, links to the right Shopify and QuickBooks API docs, and one acceptance test. A $2,000 payout with three transactions becomes three invoices in QuickBooks, each carrying the Shopify ID, total, tax, discount, shipping, and a link back to the order. Reads complete. Reads like it’s ready to hand to an agent.

The pass came back with 27 scenarios. Here are three full rows, and next to each one, my honest guess at what an agent would ship if the spec stays silent.

The 😁 line itself. Scenario: sum of transactions ≠ payout amount. Expected behavior from the pass: halt booking for that payout, mark it “Reconciliation failed” in the admin UI with both figures and the delta, notify the admin, and critically, do NOT book the invoices that happen to match and silently skip the rest. What an agent ships instead: a log line and full speed ahead, because “something is wrong 😁“ contains no instruction to stop.

The refund. Scenario: a $2,000 payout that’s really $2,300 in charges minus a $300 refund. Expected: the refund becomes a QuickBooks credit memo, and charges plus refunds plus fees still reconcile to the payout total before anything is written. What an agent ships: a $-300 “invoice,” or the refund silently skipped, because the spec’s only test contains three clean charges and nothing else.

The tag. Scenario: an admin hand-adds a “Booked” tag to an order the connector never processed. Expected: the tag is never read as the source of truth, the connector keeps its own processed ledger, so the mistagged order still gets invoiced. What an agent ships: the tag as state, because the acceptance test checks that the tag appears and it’s the only state signal the spec ever mentions. One fat-fingered tag and that order drops out of invoicing forever, silently.

Three more misses came out of the same run, and they rhyme: a re-run creating duplicate invoices because nothing demands idempotency, an auth failure leaving a payout half-booked with no checkpoint, and NZD amounts written into a USD QuickBooks file because the test never names a currency. Six gaps total, and the spec’s one acceptance test, the clean $2,000 payout, passes straight through every one of them.

Here’s the objection I’d raise myself: a good senior engineer would catch most of these. Idempotency, rate limits, token refresh, that’s table stakes for anyone who’s built an integration. If the bar is “insight no engineer could produce,” this pass doesn’t clear it, and I don’t think any checklist ever will.

That was never the job, though. The job is that this table exists before refinement, produced by the person who wrote the spec, instead of surfacing one gap at a time across three refinement calls, or worse, not surfacing at all because the reader was an agent and agents don’t snort at emojis. The pass took twenty minutes. The alternative was six defined-behavior decisions getting made implicitly, by a code generator, at 2 a.m., inside somebody’s accounting system.

And the misses aren’t random. Every one of them is the same question wearing different clothes: what happens when the money doesn’t add up the way the happy path assumed. If I were to theorize about why the spec missed all six, I’d guess it’s because the happy path only ever contains clean charges, and real payouts contain refunds, fees, adjustments, and someone’s fat-fingered tag.

So before the next spec goes into refinement, or gets pasted into an agent’s context window, run the eight categories against it. You walk away with something concrete to send: a comment to the eng channel naming the scenarios the spec didn’t cover, asking for defined behavior on each. Skip that comment and the same gap still surfaces later, mid-call, with your name on the spec. You end up saying “let’s take it offline” about a document you wrote yourself, which I’ve done, and it’s not a good feeling.

Honestly, the hard part isn’t knowing edge cases exist. Everyone knows edge cases exist. The hard part is sitting down and running the full eight-category pass on every spec when refinement starts in forty minutes.

PM OS ships that pass as a ready skill that runs on your company context, so the scenario table comes back shaped by your product and your users, not a plain checklist.

The spec-hardening layer sits next to the workflows for strategy, research, and decisions.

AI has made building cheaper, but that does not make validation obsolete. It makes validation cheaper too.

The trap is thinking those are opposites. If a prototype costs one afternoon instead of one sprint, the answer is not to skip the evidence step. The answer is to run more evidence steps, build smaller tests, and learn before the polished thing becomes too socially expensive to kill.

The PM job changes from “protect engineering from waste” to “turn cheap builds into clear evidence.”

That is a much harder job than it sounds.

The old product mistake was obvious: spend a quarter building a feature nobody needed.

The new mistake is sneakier. A PM, designer, or founder can spin up a working demo in Cursor, Claude Code, Lovable, or whatever tool happens to be hot this week. The demo looks real enough to share, gains momentum, and gets people reacting to the artifact instead of the assumption behind it.

That is how a team ships faster and learns the wrong thing faster.

I do not think the answer is “stop building.” That is dumb. Building is one of the best ways to learn, and the cost curve really has changed.

The answer is to say what the build is supposed to prove before anyone starts building.

If you cannot name the belief the build is testing, the build is not validation. It is a vibe with a UI.

There are two kinds of building now.

In build-to-ship mode, you make the thing you intend to put in front of customers, support, maintain, explain, instrument, and defend.

In build-to-learn mode, you make the smallest artifact that can test one belief.

AI makes both cheaper, but they are not the same motion. A build-to-learn artifact can be ugly, manual behind the scenes, narrow in scope, and dead by Friday. Its job is not to become the product. Its job is to answer a question.

That question has to come before the artifact.

This is where PMs are about to get sloppy. Because the build is cheap, the thinking feels optional. But cheap work still compounds. Ten cheap builds pointed at fuzzy beliefs can create more noise than one expensive build pointed at a clear one.

Cheap code raises the validation bar because you can afford to validate more.

Pull the feature out of roadmap fog and write it as a sentence that can be wrong.

Use this shape:

“We believe that [specific buyer or user] will [do a concrete thing] because [reason], within [time].”

The vague version is only a feature label:

“Build an AI weekly executive digest.”

The testable version says:

We believe mid-market operations leaders will pay an extra $15 per user per month for an AI-generated weekly executive digest because it saves them from manually preparing status reports.

Now the team can stop admiring the feature and start testing the belief.

The feature is not the unit of validation; the belief is.

Most roadmap beliefs carry one of four risks.

Desirability: do people want it or intend to use it
Feasibility: can we technically or operationally make it work
Viability: will someone pay, fund, approve, or support it
Usability: can a person use it in real work

The risk decides what kind of build or signal you need.

If the risky belief is usability, a rough clickable prototype might be perfect.

If the risky belief is viability, a polished demo might be a distraction. You may need five economic buyer calls with a price, an ROI model, and a budget-path question.

If the risky belief is feasibility, talking to users will not save you. You need a spike, benchmark, data audit, vendor test, or mock integration.

If the risky belief is enterprise adoption, a landing page click is weak evidence. It does not pass through procurement, security review, legal, setup work, or a budget owner.

This is the hidden step. Cheap building tempts you to start with the artifact. Good validation starts with the risk.

You need one page before you open Cursor.

Call it a Build-to-Learn Brief. It is a stricter version of the Efficient Signal Brief from the /work-backwards skill.

It carries:

the belief in “we believe that” form
the risk type: desirability, feasibility, viability, or usability
the cheapest signal that can see that risk
whether the signal requires a build, a call, a benchmark, a concierge test, or no build at all
the method and sample
the timebox and cost
the pass/fail threshold
the data captured
the next evidence rung if it passes

That page protects you from the new AI-native mistake: building a convincing thing before you know what evidence would matter.

A good signal clears five bars. It is early, cheap, attributable to one assumption, decisive because it has a threshold, and suitable for the market.

If a build fails any of those, it may still be useful. It is just not validation yet.

Take the AI weekly executive digest as the test case.

Engineering used to estimate eight weeks. Now a PM and engineer can probably fake a first version in a day or two with existing data, a prompt, a scheduled job, and a thin UI.

Because the first version is now cheap, the question changes from “can we afford to build this?” to “which part should we build to learn the most?”

The belief says:

We believe mid-market operations leaders will pay an extra $15 per user per month for an AI-generated weekly executive digest because it saves them from manually preparing status reports.

Classify the risk. This is not primarily feasibility. The team can probably generate a digest. It is not primarily usability either. The risky belief is viability: will a buyer pay for this, and does it fit their budget path?

So the best first signal may not be a full prototype. It may be:

5 economic buyer calls
a lightweight ROI calculator
2 manually generated example digests
one budget-path question
a threshold set before the calls

The pass line:

3 of 5 operations leaders confirm the reporting work costs more than $2,000 per month in labor time, say $15 per user per month is plausible, and name the person who would approve the spend.

The timebox:

5 business days
under $300 in incentives
2 manually generated digest examples
no production code

That is not anti-building, because the examples are still a build. They are built to learn one thing.

And here is the failure it catches: buyers love the digest, but three of five say it should be included in the base product. The team does not learn “nobody wants this.” It learns “do not build this as a paid add-on yet.” That is a better lesson than shipping a polished feature and discovering the pricing mistake after launch.

When building was expensive, teams rationed validation because every serious test felt like a project. Now the cost of a test is lower. A PM can create a mock flow, one-off report, fake settings page, benchmark script, concierge workflow, sample digest, or customer-specific demo in hours.

Lower test cost should mean more validation, not less.

Run five small tests instead of one big bet. Test viability with buyer calls and a price. Test usability with a task mockup. Test feasibility with a benchmark. Test desirability with a concierge artifact. Test adoption with a sandbox and a champion.

The win is not that AI lets you skip the work before the roadmap review.

The win is that the work before the roadmap review can now be real.

This move is not only for other people’s roadmap items.

Every PM has pet bets. You collect Slack threads, buyer quotes, competitor screenshots, and half-formed “we should” notes until the idea feels too alive to kill cleanly.

Cheap building makes that worse. You can make your pet idea look real before you have tested whether it deserves to exist.

Run the brief on your own belief before you pitch it. Write the belief. Name the risk. Pick the smallest signal. Set the threshold that would make you drop it.

Most pet bets should die at the one-week signal stage. That is not a failure. That is the new luxury. You can afford to learn before you recruit a team, a roadmap slot, and your own ego.

For PM leaders, the same artifact becomes a quality gate.

A roadmap item without a Build-to-Learn Brief is not ready for scope.

Not because building is expensive. Because building is cheap enough that teams can now create endless convincing artifacts without creating much evidence.

The hard part is not writing the brief. The hard part is the judgment inside it: classify the risk, generate several possible signals, reject the tests that lie in your market, and set a threshold before the evidence arrives.

AI makes that judgment cheap enough to repeat across the roadmap.

It can take one belief, your segment, your constraints, your assets, and your timebox, then produce five candidate signals. It can tell you when a build is useful, when a call is better, when a benchmark is required, and when a fake-door test is nonsense for the buyer process.

You still own the call. You edit the threshold. You know the customer and the politics.

You are no longer starting from a blank page or a generic “validate this idea” prompt.

That is the PM OS angle. The value is not another prompt. It is reusable PM judgment: classify the belief, choose the signal, set the threshold, and turn the work into an artifact you can use in review.

The new rule is simple: when building gets cheaper, validation should get more frequent.

Do not stop at “can we build this quickly?” Ask, “what will this quick build prove?”

Pick the roadmap item you are least sure about. Write the belief. Name the risky assumption. Decide whether the best signal is a call, benchmark, concierge test, rough build, or no build at all. Set the pass/fail threshold before you start.

If you cannot answer those lines, the work due before the next build is not another mockup or another prompt. It is the Build-to-Learn Brief.

The structured version of this is /work-backwards inside PM OS. Install it, paste one belief, add your segment, stage, constraints, assets, and timebox. It gives you the signal, threshold, next evidence rung, and brief you can take into planning.

Ships pm-os-agent-builder — PM OS 2.5’s headline skill. A meta-skill that helps users author their own custom agents for whichever harness they run (Claude Code, Cursor, Cowork), grounded in live-docs research on how agents actually work in each.

I researched all three harnesses’ agent mechanics + long-running/recurring/goal execution, then distilled it into the skill. Two learnings drive the design:

The “Fit” gate — a subagent only earns its context via parallelism, protecting the main thread, external MCP reach, or cadence.
Cadence — recurring work becomes a schedule/loop, not a self-looping subagent (a subagent cannot ScheduleWakeup / self-schedule).

SKILL.md — 7 steps: Harness → Fit → Job → Cadence → Tools & MCP → Draft → Place
references/harness-specs.md — per-harness frontmatter, paths, upgrade-safe homes, gotchas
references/cadence.md — long-running / recurring / goal-driven, all three harnesses
references/idea-gallery.md — 11 curated agents that earn their context, harness-tagged (+ 3 explicitly demoted to “command, not subagent”)

Agents are placed where /upgrade can’t touch them: ~/.claude/agents/ (Claude Code), .cursor/agents/ (Cursor), a standalone user plugin (Cowork).

I ended up going through almost all of my skills and updating them for the latest advice around how skills should work & how harnesses work.

I feel like there’s still a bit more work to do in terms of making them even better, and I think that’s probably gonna come in version 3.0 eventually.

Install AI PM OS and get building

🕹️ Working Backwards and Catching Edge Cases With PM OS

Comments