RSS Amplifier

The Operator's Notebook · Nov 29, 2025

The AI Hallucination Debate Is Missing the Point

0
Sign in to vote or save

Max Braglia · The Operator's Notebook

Last week, I posted something on LinkedIn that got 166 comments and made some people genuinely angry.

I said: “I love when people complain about AI ‘hallucinations’ like it’s some moral failure. It’s a tool. You verify what it spits out — exactly like you double-check your intern’s draft, your freelancer’s research, or your employee’s numbers.”

The responses came fast. Some people loved it. Others called me out for comparing humans to disposable tools, for not understanding how AI actually works, for being dangerously naive about the technology.

Almost nobody engaged with my actual point.

Instead, I got five completely different arguments from five completely different groups of people. Developers agreed with me. Translators told me I was destroying livelihoods. Engineers explained why I was technically wrong. Educators worried about societal impacts. Skeptics said “it depends on the context.”

And here’s the thing: reading through every single comment, I realized they were ALL right.

Not because AI is inherently good or bad. But because each group was talking about completely different problems, completely different use cases, and completely different risks.

The hallucination debate we’re having? It’s a distraction from the conversations we actually need to have.

Let me be clear about my original argument.

AI produces imperfect output quickly. Your job as the user is to review it, fix it, and ship it — just like you would with work from any team member. This is a process issue, not a technology crisis.

This approach works exceptionally well for specific situations:

  • Content drafts where you already know the subject matter

  • Code scaffolding where you can test the output

  • Research starting points where you plan to verify anyway

  • Reformatting, summarizing, and brainstorming

Here’s a real example from my client work. I built a content system for financial advisors who need to publish marketing content under strict SEC and FINRA compliance rules. The workflow looks like this:

AI drafts the article → compliance review agent checks it against regulations → human editor reviews and adjusts → final compliance check → publish.

What used to take weeks and cost thousands in legal review now takes a couple of days and costs a fraction of that. It works because we built verification into every step of the process.

The key principle: if you’re using AI without verifying the output, that’s your process failure. Not the AI’s moral failure. Not a technology crisis. Your workflow problem.

That was my point. But clearly, people heard something very different.

My post triggered five distinct reactions, and understanding each one matters more than being right or wrong.

Developers, marketers, and business owners who use AI daily immediately got it. They’ve built systems and workflows around AI. They understand it’s a tool that requires oversight.

One commenter put it perfectly: “People treat ‘hallucination’ like the model betrayed them, when really it just means you didn’t verify the draft.”

Their frustration isn’t with AI. It’s with people who ship unverified AI output and then blame the technology when it fails.

Translators, writers, and junior employees heard something completely different in my post. To them, comparing “interns, freelancers, and employees” to AI tools felt dehumanizing.

One translator wrote: “So your interns, freelancers and employees are ‘tools’ as well... Wonderfully humane, farsighted and enlightened take.”

The heat in these responses isn’t really about the comparison. It’s about the very real fear of being replaced by technology that costs less and works faster.

And here’s the concern I initially missed: if businesses adopt “AI for junior work, humans only for verification,” where do future senior experts come from?

Junior roles exist to build expertise. They’re an investment in human capital. If AI does all the junior work, you’ve eliminated the entire pipeline of developing talent.

This isn’t theoretical. I’m building systems right now that make entry-level marketing roles unnecessary. That’s real. And I don’t have a good answer for what comes next.

Several engineers and technical professionals pushed back on my comparison between AI errors and human errors.

One wrote: “A reliable intern wouldn’t invent non-existent facts. AI can hallucinate unpredictably, and you essentially have to review everything.”

Another pointed out: “AI errors aren’t like human errors. Humans make mistakes in areas where they lack knowledge. AI makes up facts with complete confidence in areas where it has no knowledge at all.”

This is the critical distinction I glossed over: a junior employee who fabricates sources or invents data gets fired immediately. That’s not a mistake — that’s an integrity violation.

But AI does this by design. It’s not trying to deceive. It’s a probabilistic system predicting the next most likely words. It will confidently cite “Johnson v. Smith, 2019” when no such case exists, not because it’s lying, but because those words had high probability in that context.

You can’t fix this with better prompting. You can’t train it away. It’s fundamental to how large language models work — they predict probable next tokens, not truthful ones.

The engineers are right: this IS different from human error.

Some commenters raised concerns I hadn’t even considered.

An education professional wrote: “AI proponents are creating dependency in a population that lacks the requisite independent critical thinking skills to verify the output.”

Others pointed to growing research about AI-induced psychological issues. About people using AI for legal advice, medical decisions, and financial planning without the expertise to catch dangerous errors.

Their (good) point: not every verification problem is solvable. If someone uses AI specifically because they lack expertise in an area, they literally cannot verify whether the output is accurate.

The verification workflow I described only works when you already have enough knowledge to catch the mistakes.

The most thoughtful pushback came from people who said “you’re right for some things, dangerously wrong for others.”

One commenter shared: “When I have to deeply fact-check AI output, it’s often easier to just do the research manually. I learn more that way too.”

Another mentioned the lawyer who used AI to research case law, and the AI invented multiple court cases that didn’t exist. The fabrications were convincing enough that they made it into a legal filing.

Their point: the “verify everything” approach works great in some contexts and fails catastrophically in others.

Here’s what became clear reading all these responses: everyone’s debating “are hallucinations acceptable or not?”

That’s the wrong question.

The better question is: “For what tasks is AI appropriate, and for what tasks is it dangerous?”

Because here’s the verification paradox: if you already have enough expertise to verify AI output, you probably could have done the work yourself. If you lack the expertise to verify it, you shouldn’t be using AI for that task at all.

This isn’t about AI being good or bad. It’s about matching tools to appropriate tasks.

I’ve started thinking about AI use in three zones:

Green Zone – AI Works Great:

You have expertise in the domain. Verification is significantly faster than creation. Errors are obvious and easy to catch.

Example: I can use AI to draft marketing content about digital marketing strategies because I’ve been doing this work for 25 years. I can spot bad advice instantly. The AI saves me time on the initial draft, but I’m bringing the expertise.

Yellow Zone – AI Needs Serious Guardrails:

You have some expertise, but verification takes real work. Errors might not be immediately obvious. You need structured systems.

Example: Using AI to help with code in a programming language you’re learning. You can test the code and catch some errors, but you might miss subtle problems or bad practices.

Red Zone – Don’t Use AI:

You lack the expertise to verify output. Errors could cause real harm. Verification is actually harder than just doing the work yourself.

Example: Using AI for legal research when you’re not a lawyer. Medical diagnosis when you’re not a doctor. Financial analysis when you don’t understand finance.

Most of the AI criticism I see comes from Red Zone usage. Most of the AI success stories come from Green Zone usage. And most of the arguments come from people in different zones talking past each other.

I’ve been thinking hard about the pushback, and some of it landed.

The critics got these things right:

The human capital problem is real. Junior roles exist to build future senior experts. If AI does all the entry-level work, where does the next generation of expertise come from? I’m literally building systems that eliminate these roles, and I don’t have a solution for this yet.

The dehumanization concern is valid. Comparing “interns, freelancers, employees” to AI tools revealed something about how modern business views labor. The comparison is mechanically accurate — you do verify their work — but it’s philosophically uncomfortable. People aren’t just production units.

The technical distinction matters. AI hallucinations ARE structurally different from human mistakes. They’re non-deterministic, unpredictable, and baked into how the technology works. You can’t fix them through training or feedback. You can only work around them.

But here’s what I still stand by:

Process matters more than technology. If you’re using AI without verification systems and something goes wrong, that’s on you. The tool is doing exactly what it was designed to do. Getting mad at it for hallucinating is like getting mad at a hammer for not being a screwdriver.

The outrage is often misplaced. People who expect perfection from AI don’t understand what it actually is. It’s not intelligence. It’s pattern matching. It’s not knowledge. It’s prediction. And there’s genuine value when used appropriately.

We need to move beyond the binary thinking of “AI good” versus “AI bad” or “embrace it” versus “ban it.”

Instead, we should be asking: “What work should humans do, what should AI assist with, and what should AI never touch?”

For businesses, the questions are:

Are we using AI in Green, Yellow, or Red zones? Do we have verification systems actually built into our workflows? Are we eliminating entry-level positions that develop future expertise, and if so, what’s our plan for building talent?

For individuals using AI:

Am I using this as a tool or as a crutch? Can I actually verify what it produces? Am I building skills or outsourcing thinking?

For society broadly:

How do we develop expertise if AI handles all junior work? What safety nets exist for workers displaced by this technology? What regulation makes sense for high-risk AI use in fields like healthcare, law, and finance?

The nuanced take isn’t sexy, but it’s honest:

AI is exceptional for drafting, iteration, and pattern matching in domains where you have expertise.

AI is risky for fact-finding, expert analysis, and anything requiring truth verification.

AI is inappropriate when verification is harder than just doing the work yourself, or when errors could cause serious harm.

The hallucination debate is a distraction from more important questions.

The real conversation is about appropriate use, verification systems, and long-term consequences we’re not thinking through carefully enough.

Both the hype and the fear are wrong. The answer is in the middle, and it’s more interesting and more complex than either extreme.

If you’re using AI: ask yourself honestly what zone you’re operating in. Green, yellow, or red. Be honest about whether you can actually verify the output.

If you’re building AI systems: build verification in from the start. Make it impossible to skip. Design for the worst case, not the best case.

If you’re worried about AI: some of your concerns are absolutely valid. It depends entirely on the context and how it’s being used.

Technology doesn’t have values - we do, and that means deciding what we should do with AI matters more than debating what it can do.

No posts

Read the original on maxbraglia.substack.com

Comments

Nothing yet. Say the first thing.

    Sign in to join the conversation.