Coding Droplets

Thanks!

Hi Kartik, Thanks! You picked the part I'd underline too. The correlated-failure point is exactly why one check isn't enough. If the relevance filter uses the same embeddings as the retriever, it agrees with the retriever's mistake by construction. Second layer has to be judging something different. On citations, the scores earned their keep in support more than in the product. When someone reports a bad answer, the scores tell you instantly whether it was retrieval or generation.

Hi Kartik, Thanks! Threshold tuning is the part most people skip. No fixed number. The distribution shifts with the embedding model, so I shadow-run the cache for a week, log top scores without serving anything, then label a few hundred real pairs and find where false hits fall off. On tracking, rephrase rate is the giveaway. If people re-ask more after a hit than a miss, the threshold's too loose. Thanks again for sharing yours.

Yes, exactly this. "Earn your way to an agent" sums up the whole article better than I did honestly. The whiteboard test has never failed me - if you can write the steps down in a fixed order, you just proved you don't need the loop. Thanks for reading!

Thank you! That's a great point and it's actually something we considered adding as a dedicated section, though the post was already running long. Industry context does change the calculus significantly. Healthcare is a good example: if you're working with FHIR, the specification already defines its own error format (OperationOutcome), so Problem Details effectively gives way to the domain standard in that context. Fintech is another case worth noting. Those teams typically adopt Problem Details but rely heavily on its extension fields, largely because audit and traceability requirements demand machine-readable error codes that align with regulatory expectations. High-traffic domains like retail, on the other hand, tend to favor the minimal HTTP-semantic approach, since CDN-level caching delivers meaningful cost savings and an envelope tends to work against that. The decision framework still applies in each case. It's the weighting of the criteria that shifts depending on what you're building. The one constant across every industry is that the "200 OK with success: false" pattern should always be avoided.