How to Argue With a Chatbot

Something I’m hearing executives say a lot right now is, “I use AI to challenge my thinking.” But what does that look like? When I ask, I’m told something along the lines of, “Well, I ask it my question, read its response, and consider what it has to say.”

Sometimes they’ll ask a follow-up question, but they don’t push back. Every time I’m left wondering, “Where’s the challenge in that?”

The Agreeable Machine

The reality of AI is that it’s trained to be helpful and conversational, but it operates without consciousness and on pattern recognition. This means that it can quickly become dangerous even if you’re trying to use the tool for strategic thinking.

When you share an idea with a chatbot, its default is to validate and expand upon it with supporting evidence that develops an argument in your favor, re-packaged with a bow. You walk away feeling productive and accomplished, but all you really got was a confirmation loop complete with your biases absent of challenge.

This confirmation loop is at work in every boardroom and inbox where AI has either knowingly or unknowingly replaced thinking.

Ignite an Argument

The fix is to ignite an argument. Yes, you heard me right – ignite an argument. To truly use AI to challenge your thinking, you need to use it to pressure-test your ideas by asking questions like:

What would have to be true for this answer to be completely wrong?

How does this recommendation fail in a market crash/PR crisis/supply shortage?

What perspective is missing from this output?

When you ignite an argument and make questions like these part of your workflow, AI goes from being a useless validator to a stress-testing machine that has the power to 10X your success.

Any outputs are suddenly the starting point to a true conversation that actually challenges your thinking beyond the screen.

Invite Discomfort

Oftentimes we don’t ask these questions because, subconsciously, it can be uncomfortable to hear the answers. But asking questions like these prevents us from falling into the alluring and costly trap of technology that’s designed to be agreeable, making it the most important leadership practice of today.

Think about it this way: they’re questions any good board member or investor would ask anyway.

Yes, answers to questions like these dismantle our strategy and feel like a punch to the gut. As humans, we’re wired to appreciate confirmation more than anything else. It makes us feel good, and it’s why nearly every leadership framework tries to counteract it.

Why? Because leaders who find disagreement as a signal to push harder come out ahead. Leaders who do the same with AI are no different. When you test the emergency systems while things are calm, they work when it’s not.

The Test of Genuine Thinking

How do you measure the quality of your interaction with AI? The answer comes in the form of a question: How often does it change your mind? If the answer is never, the tool is thinking for you, and you’re stuck in a confirmation loop under the guise of productivity.

If you’re really using AI to challenge your thinking, every interaction with the tool will feel uncomfortable.