My AI is Too Confident and I Hate It

I was in a conference room with a client a few months ago, looking at an AI-generated report. It was maybe twenty pages, polished, full of charts with clean lines and summaries that moved from problem to insight to recommendation. The thesis was confident and the structure was airtight.

Halfway through, I realized I had no idea if any of it was true.

The entire document had the appearance of knowledge without containing any residue of how that knowledge was formed. There was no uncertainty footnoted anywhere. No “we didn’t have access to” or “this data point was incomplete” or “we ran out of time to verify.” Just clean text and confident conclusions, delivered with the kind of authority that used to mean someone had sat with the problem long enough to understand it fully.

I looked at my client. He was nodding. And I realized something in that moment that I’ve been sitting with ever since: he had no idea either. Neither of us knew if the report was true. And we were both performing the act of believing it because the confidence was so clean.

Confidence used to mean something different. When someone made a confident statement, there was usually a tax attached to it. They had lived through versions of the problem and staked something on the answer. Their confidence had been earned through the specific accumulation of their experience, and when it failed, they felt the impact.

There was asymmetry built in. The person making the claim carried more risk than the person hearing it. That imbalance kept people honest.

Now confidence is cheap. It comes out of a system designed to never hedge, hesitate, or acknowledge the gap between what it has seen in its training data and what is actually true in the moment in front of you. The system generates confidence as an artifact of how it works. Confidence, as the system produces it, is simply a default setting.

And when everyone in the room has learned to read confidence as a signal of certainty, the collapse happens so quietly that you don’t notice it’s happened until you’re sitting in a conference room nodding along to something you don’t believe, watching someone you respect do the same thing.

Confidence used to mean something different. When someone made a confident statement, there was usually a tax attached to it.

The report probably wasn’t wrong in any dramatic sense. It probably contained patterns that were real enough. The issue wasn’t whether any specific claim was false. There was simply no way to know where the edges of its knowledge were, which claims came from solid data and which came from inference, whether the recommendations fit the client’s actual situation or matched a pattern the model had seen a thousand times before. The confidence obscured all of that.

A system trained on massive amounts of human writing learns the texture of authority. Confident voice sounds like answers, not questions. Hedging signals weakness. Specificity and certainty are so deeply wound together in the corpus that generating one without the other reads as broken. So the system generates them together, always, as a learned artifact of what authority sounds like in the training data.

This creates a peculiar kind of trap. The model produces work that reads as authoritative because authority, in the training data, is primarily a stylistic feature. It has texture. The connection to epistemology, to actually knowing the thing being stated, is not part of the output. The human on the receiving end is trained to read confidence as credibility because confidence used to require accountability. The gap between those two things is where we’re living now.

For years, I kept asking the same question in rooms full of people deploying AI systems at scale:

How do you actually measure the quality of the output?

The room would go quiet. The kind of quiet so awkward that everyone exhales slightly because someone finally said the thing they had all been thinking at some point. Everyone had tried to answer the question and found the same thing: the confidence of the output had nothing to do with its reliability. By the time you’d invested enough effort to verify whether something was true, you had already defeated the purpose of using the tool.

What we’ve lost is the tax on confidence. And with it, the signal that used to keep authority tethered to accountability.

The person who made a confident claim had to live with the consequences of being wrong. That cost mattered. It focused attention and rewarded accuracy over fluency. Confidence used to be legible in a specific way: when someone had checked, you could tell. When they were hedging because they hadn’t yet, you could tell that too. Those signals were readable in a way that mattered.

Now the system has no relationship to consequences, no memory of being wrong in ways that matter, no skin in the game. It generates confidence the same way it generates anything else, as a pure artifact of how the machinery was designed to do.

The people using it have started to internalize the same carelessness. If a claim arrives with enough authority to function as truth, verifying it feels like extra work. Slowing down to check undermines the whole point of using the tool. And when the system itself carries no responsibility, taking responsibility for what gets deployed starts to feel optional.

The confidence is still there. It’s just no longer tethered to anything.

What stays with me about that conference room is not the report itself. It’s the moment when I looked across the table and saw someone I respect nod at something neither of us believed.

It takes more energy to resist the performance than to participate in it. Saying out loud that you’re uncertain about something that sounds certain costs something. The person who accepts the report and nods reads as more decisive, more professional, more ready than the one who says “I don’t actually know if this is right.” In a company racing to deploy something, “I don’t know if we should trust this yet” is a tax on momentum. So people don’t say it.

Instead, they nod. And the confidence propagates down the chain. By the time the recommendation reaches someone with actual authority to make a decision, the fact that no one in that chain has actually verified anything has been completely lost. What remains is the echo of confidence, now layered three or four times, each layer a little bit more certain than the last.

I keep thinking about this in the context of work. The real scarcity used to be information. Now information is infinite and cheap. The real scarcity is judgment. But judgment requires something that confidence, as currently generated, actively erodes: the willingness to say “I don’t know,” to hold uncertainty, to carry the weight of stakes. It requires people with enough standing in the room that they can afford to be wrong and still be heard.

And those people are rare. Rarer, maybe, than they’ve ever been.

I don’t know how to unsee this once you’ve seen it. Once you’ve noticed that the person across the table is performing confidence they don’t feel, it becomes harder to trust that anyone is actually grounded in what they’re saying. The absence of verification becomes audible underneath everything. Every conversation carries the faint sound of something that should have been checked and wasn’t.

One answer is to build systems where the default output is constraint rather than conclusion, where “we don’t know” is visible when it’s true, where the cost of uncertainty shows up in what the system produces instead of being papered over.

The work we’re building at SmartHaus is an attempt at something like that. The premise is that if constraints can be measured at every touchpoint in a workflow, if a system’s exact authorization can be defined and made visible, then confidence gets reattached to something verifiable. Authority becomes trustworthy again because the limits governing it are mathematically defined.

Right now, I’m sitting in rooms with reports that are more confident than they should be and people who are more uncertain than they’re performing.

The distance between those two things is where the real work is. The gap I keep coming back to. What I’m still sitting with.

The version of confidence that survives will be the kind that has something to lose. Everything else is just style.

Rethinking authority, Yen.