You Turned Your Agent Into a Fabrication Engine

Domain 4 · Lesson 4.4 · Structured Output and Grounded Verification

tl;dr Asking again is not a way of checking an answer. When the thing you asked for was never in the source, every repetition is pressure to produce one, and you will get something that looks completely ordinary.

You already do this by hand. An answer comes back, something about it feels off, and you type “are you sure?” or “check that again.” Often enough it corrects itself, so the habit sticks.

There is a sharper version of the same move, and I see people make it constantly. You ask for the answer it did not give you. What was your second choice, what else did you consider, what is the next best option.

That sounds like retrieval. It sounds as though the model lined up its candidates, picked one, and is holding the runner-up somewhere you can ask for it. Nothing was held back. There is no shortlist sitting behind the answer. And the question has already told it that another answer exists, so “there isn’t one” is no longer something it can say back to you.

Here is what that costs, in the only setting where I actually care about it. My health agent reads doctor’s notes and lab results, and about one note in twelve records a medication with no dose written down anywhere on it. Ask a system built the wrong way for that dose and it returns one. It comes back as 50mg. It is formatted correctly, it is a plausible number for that drug, it sits in the right field, and nothing anywhere errors. It is also not in the note, and there is no line in any log that will ever tell you so.

And the cost is not one bad field. It is the whole record. If the agent can produce a dose that was never written down, and nothing in the output separates a value it found from a value it made, you have no way to tell which of the other values are real. They are all now “probably right,” which is a different thing from right, and on a health record it is not a category you can make a decision in.

Everything below the line is the fix: the one question that tells you whether a value was found or produced, why the same repair makes things worse on one kind of failure and better on another, and the mismatch that your own code can detect but cannot resolve. It ends with a prompt you paste into Claude Code that goes through your own health agent, finds every field that can quietly invent a value, and rewrites it so the agent says “not stated” instead. Your records, your agent, your machine.

Before you decide how to fix a wrong value, answer this: is the information in the source and reported wrongly, or is it not in the source at all? Everything in this lesson follows from that. On the first side, asking again works, because there is something there to find. Feed the model the original note, the answer it gave, and the specific reason that answer failed, and it will usually correct itself. Specificity is what makes it work. “Field dose was empty but the note contains ‘take two 250mg tablets’” is a repair instruction. “Try again” is a nudge and performs about as well as one.

On the second side, asking again is the worst thing you can do. There is nothing to find, the model has been asked to produce something anyway, and each repetition raises the cost of returning nothing. Retrying for information that is genuinely absent is not a check. It is a fabrication engine, and the more polite and persistent you are, the better the invention gets.

That is also the honest answer to the next-best-answer question from up top. The move is not always wrong. If the lab report lists twelve analytes and the agent surfaced one, asking what else was on the panel is fine, because the other eleven are sitting right there in the document. The move goes wrong when you cannot point to where another answer would live. And the person asking usually cannot, because if they could see what was in the document they would not be asking the model.

So the repair is the same one that fixed the fabricated purchase order back in Lesson 4.1, and it works identically in a schema and in a chat window, because in both places the problem was that you had removed the option of saying nothing. Give absence a legal way to come back to you. In a schema, that means the field is nullable and dropped from the required list. In a chat window, it means one clause:

Not: “what was the next best option?”

Instead: “is there another option stated in the document, and if there is not, say so explicitly.”

That clause costs you nothing, and it restores the answer you actually needed, which is “none.” Nobody will ever thank you for it, because the reply you get back is a shrug, and a shrug from a health record is the correct answer more often than people expect.

Retry used to carry a much bigger load. If your extraction came back as malformed data, you caught the parse failure and asked again, and often that worked.

That job is gone. Structured outputs, from the last lesson, make malformed data impossible rather than unlikely. Shape, field types, which fields are mandatory: all of that is now guaranteed before the answer is produced, and retrying to fix it is spending a round trip on a problem that can no longer occur.

What is left for retry is the meaning of the values, not their form. Whether the numbers agree with each other. Whether the extraction contradicts itself. Whether a cross-field rule holds. A structured output will happily hand you {"total": 1500} as a perfectly valid number when the line items add to 1200, because the rules you wrote described what a total looks like and never described what a total is.

That is the remaining job, and you should know you are down to it. A retry loop written three years ago is probably still guarding against problems that can no longer occur, while missing the ones that can.

This is the part I got wrong, and if you take one thing out of this piece I would like it to be this. Have the agent extract the count of medications alongside the list of them, and have your own code compare the two. That is a good check, and it is cheap. The note says five medications. The list has four. Your code flags it in a millisecond, and then you are stuck, because you cannot tell from the mismatch alone which side is wrong. Either the count is bad, in which case asking again fixes it and everyone goes home, or a medication was dropped during extraction, in which case the fifth one is absent from what you captured, and asking again will produce a fifth medication that was never on the note. Same signal. Opposite handling. And the check that found the problem has no way to tell you which one you are looking at.

That is the trap in this whole domain in one example. A mismatch that looks semantic can be an absence problem wearing a disguise. Deterministic code can count the length of a list. It cannot tell you whether the list is complete.

The rule that falls out of it: when your check can identify which value is wrong, retry with that specific error. When it can only tell you that two things disagree, escalate to a person. On a health record that means a flag and a human, not another attempt.

A conversion you could just do. One in three of my lab dates arrives as 03/04/2026 when the field wants 2026-03-04. Retry does fix this. It also costs a model round trip, adds latency, and introduces a small chance of a new error, in exchange for a transformation that is four lines of code and cannot fail. Retry working is not the same as retry being right. If the repair is deterministic, do it deterministically.

Anything you have wrapped in a uniform loop. The instinct is to write “retry up to three times” once and apply it everywhere, because that is what good engineering hygiene looks like. But the missing dose should never be retried at all, the date should never reach the model, and the count mismatch needs a person. One budget across all of them is a design smell, and it comes from the same instinct as last week’s piece: reaching for one mechanism before diagnosing which failure you have.

Where you do retry, cap it at two or three attempts and then escalate. An unbounded loop against an absent value is a machine for producing confident nonsense.

Two prompts. The first one goes looking through what your agent has already extracted and tells you which values were never in the source. Paste it into Claude Code in the folder where your agent and its records live.

Go through the records my agent has already extracted in this folder and find
values it produced that were not in the source document.
Work out for yourself where things live: the extracted records, the source
documents or notes they came from, and the extraction prompt and field list.
Show me what you found before you start so I can confirm.
For every extracted value where you can see the matching source document, check
whether the value actually appears there. Report each one as:
  FOUND       the value appears in the source, quote the line it came from
  DERIVED     the value is not written but is computable from what is written,
              show the computation
  NOT PRESENT the value is in the record and nothing in the source supports it
  UNCHECKABLE you could not locate the source document for this record
Do not guess on behalf of the source. If a value is a plausible one for that
field but the document does not state it, that is NOT PRESENT, and it is the
whole point of this exercise.
Give me the NOT PRESENT list first, grouped by field, with the count for each
field and the source document for each value. Then tell me how many records were
UNCHECKABLE, because that number is the part of my data I cannot say anything
about either way.

If your agent kept the source text next to what it extracted, this produces a list you can read. If it did not, everything comes back UNCHECKABLE, and that is its own finding: you have a record you cannot audit, and the second prompt matters more, not less.

Read the NOT PRESENT list before you fix anything. In my own records the fields on it were not the exotic ones. They were the ordinary fields I marked required months earlier, without once thinking about what happens when the note simply does not say.

The second prompt makes the change and proves it worked.

Change my extraction agent so that a value which is not stated in the source is
reported as missing instead of guessed, and then prove to me that the change
works.
Find the extraction prompt and the schema or field list yourself and show me both
before changing anything.
First, for every field, tell me whether a real source document could plausibly
lack that value: a medication with no dose written down, a lab result with no
reference range printed, a date that is simply not on the page. List the fields
that can be absent but currently have no way to report absence, and wait for me
to confirm the list.
Then, for each field I confirm:
  - make the field nullable and remove it from any required list
  - add a companion field recording why it is empty, from a fixed set of reasons
    such as "not stated in source" or "unreadable"
  - add a line to the extraction prompt saying that when a value is not stated
    the agent must return the empty value and the reason, and that a plausible
    guess is a failure and not a fallback
Then prove it. Take one real source document from this folder. Make a copy with
one of the confirmed values removed from the text. Run the agent on the original
and on the copy, and show me both results side by side. The original should carry
the value. The copy should come back empty with a reason, not with a number.
If the copy still comes back with a value, the change did not work. Say so
plainly and tell me what you would try next. Do not adjust the test document to
make the result look better.

That last instruction is there because I have watched a model quietly weaken its own test rather than report a failure, and a test that has been edited to pass is worse than no test.

What you have at the end is not a report. It is an agent that answers “not stated” and means it, a demonstration on your own document that it does, and a list of the values already in your records that nothing ever supported. Those values have been sitting in there the whole time, and you have been reading them as facts.

A system that will produce a dose rather than report that none was written is not on a path to needing less of you. Somebody had to know that value could be missing, decide a blank was safer than a plausible number, and build the record so it could say so. That is not oversight of the machine. It is deciding what the machine is allowed to claim, and I do not expect to hand that part over.

Lesson 4.5 is next Tuesday, on running these extractions in batches, and on what changes when nobody is watching any individual result go by. Thursday takes that last paragraph somewhere else: what this failure has to do with the layoff numbers everyone has been arguing about this month. If you are new here, the map is at Start Here: The Claude Architecture, and the lesson this one leans on hardest is ‘Be Careful’ Is Not an Instruction.

Daniel Williams builds his own AI tools in the open at dewilliams.co.

Claude Code for Non-Coders publishes on Tuesdays and Thursdays. If this saved you an afternoon, send it to the one person you know who needs it.