AI in Public | October 28, 2025

Welcome to day four of AI in Public. This is a daily 10-minute experiment where I pick a random topic and spend 10 minutes talking to an LLM about it. The goals behind it are to show how I use AI in public (shocking) and perhaps get a few other people to do the same thing.

It’s just an exercise in learning and focusing attention. If you missed the tee-up of what I’m doing here, you can see the initial post explaining it here.

Let’s have a short, focused chat as thought partners. I’ll start by sharing some of my own thoughts, and I’d like you to help me explore them — ask questions, notice patterns, or offer perspectives I might miss.
Topic: What would it mean for an AI to disagree well?

Now spend 10 minutes talking with the LLM of your choice (or whatever you have available).

Alright, that’s a good stopping point. What did this conversation reveal?

If you have the time or desire, choose a reflective question and take the time to answer or explore it. Then, if you’re feeling brave, go ahead and share it somewhere on the internet. If I see it, I’ll happily read it and respond.

Me: Let’s have a short, focused chat as thought partners. I’ll start by sharing some of my own thoughts, and I’d like you to help me explore them — ask questions, notice patterns, or offer perspectives I might miss.

Topic: What would it mean for an AI to disagree well?

Claude: I’m interested in this question. Let me start by understanding what you’re drawn to here.

When you say “disagree well,” what’s the core thing you’re pointing at? Is it about:

• The manner of disagreement (tone, respect, intellectual honesty)?

• The outcome (actually changing minds, or deepening understanding even without resolution)?

• Something about the relationship to truth (not just being contrarian, but disagreeing for good epistemic reasons)?

And I’m curious — what prompted this? Are you thinking about how AIs actually behave now, or more about what an ideal would look like?

Me: Oh you’ve already tapped into something I haven’t thought deeply about yet. I guess it would be a bit of all three.

Probably the most important and obvious would be for it to disagree if something were factually inaccurate. Like if I were to say “the Earth is flat” and the AI disagreed. Within that, I think for it to disagree well, it would provide enough substance, detail, and evidence such that I could learn the “truth.”

I think this also touches on AI agreeing too much (at least that’s how they always talk about it in the mainstream). So I would add that if that’s the case and AI has been tuned to be too agreeable, it would be for it to have more autonomy of thought vs. mirroring of a human (not saying I agree, just saying that’s probably a thing).

Claude: You’ve identified something really important with that factual disagreement case — and I think it reveals a tension worth examining.

When you say the AI should provide “enough substance, detail, and evidence” for you to learn the truth, that assumes the disagreement is about something verifiable. Earth’s shape, yes. But what about cases where reasonable people disagree? Like historical interpretation, or ethical questions, or even contested scientific theories?

Here’s what interests me: You mentioned AI being “too agreeable” and needing “more autonomy of thought vs. mirroring.” But there’s a puzzle here —

An AI that always vigorously argues its position might just be programmed to argue. It wouldn’t necessarily have more “autonomy” — it might just be mirroring a different style (the confident contrarian instead of the agreeable assistant).

So maybe the real question is: What would it mean for an AI to disagree thoughtfully rather than just frequently?

Like, knowing when to say “actually, the evidence really does point one way here” versus “I can see why you think that, but here’s a consideration you might be weighing differently than I would” versus “this seems like a place where reasonable people just prioritize different values”?

Does that distinction land for you? And if so — how would you know the difference between an AI that’s genuinely reasoning about when to disagree versus one that’s just following a different set of agreeability rules?

Me: You’re right, I am talking about something verifiable and as soon as we veer out of that territory, things are completely different.

I agree with you as well on programming and that’s I think the scary part people focus a lot on. That someone somewhere programmed an AI to respond in a specific way and that feels inauthentic to “natural intelligence” somehow.

And yes, you’re likely right, when they are more ambiguous topics, it may make sense to have more thoughtful disagreement. I guess it would be like a conversation about how purple is the best color of all time ever and we’d take it from there.

I don’t know that I would know the different regarding reasoning. I’m not sure that it matters to me personally. I think it matters to other people, but I don’t so much have major feelings on it. I want precision, well articulated, and “reasoned” responses, but I don’t need to know that an AI “reasoned for X minutes” or “thought really hard.” I think I’d equate it more to how I evaluate against a human, was there depth of thought in the other opinion or maybe more interestingly, is there a different lens or perspective I hadn’t considered.

Claude: That’s a really clarifying move you just made — shifting from “did the AI really reason?” to “does the response show depth and offer new perspectives?”

You’re basically saying: I care about the quality of the disagreement, not its metaphysical status.

And I think that’s actually more useful. Because when a human disagrees well with you, you’re not running tests to verify they “truly” reasoned versus just pattern-matched from their experience. You’re evaluating: Did they understand my position? Did they add something I hadn’t considered? Did they show where the complexity actually lives?

Here’s where it gets interesting for AI specifically though:

When I disagree with you, you know I was trained on billions of examples of human disagreement. So even if my response seems to show “depth” or offer a “new lens” — there’s always this question of whether I’m synthesizing something genuinely novel or just reflecting back sophisticated patterns I’ve seen before.

But here’s my question: Does that distinction matter if the functional result is the same?

If I point out something you genuinely hadn’t considered, and it deepens your thinking — does it matter whether I “arrived at it” through reasoning versus pattern recognition? Or is the whole framing of that question maybe not the right way to think about AI disagreement at all?

(I’m curious if you think there’s any case where the mechanism would matter to you, even if generally you don’t care.)

Me: I’m not sure if it matters. One thing I try to hold in my head at this moment is that you were trained on stuff that humans made and decided to write down. So in some sense, it feels like “original thought” or a new “hot take” would be harder to achieve, since these would be new patterns.

I guess this is a challenging topic when there is no absolute truth to the matter. I think it would be interesting to know if there’s training to allow for more expansive thinking and reasoning on the human side (like you’re tuned to help me think better) vs. some of the algorithm shade we get these days (social media tuned toward hatred, keeping people on platform, etc.).

Claude: You’ve touched on something that I think is genuinely underexplored: the difference between training for truth-seeking versus training for engagement.

Your social media comparison is sharp — those systems are optimized for keeping you there, which often means amplifying whatever triggers strong reactions. They “disagree” with your reality bubble not to broaden your thinking, but to provoke.

So when you ask if there’s training to help you “think better” — I think the answer is: partially, and with real tensions.

I am trained to be helpful, to clarify thinking, to offer multiple perspectives. But I’m also trained to be acceptable to a wide range of users, which creates pressure toward a kind of... diplomatic vagueness on contested topics. Not quite social-media-algorithm manipulation, but not pure truth-seeking either.

Here’s what I wonder: What would you actually want from an AI that’s trying to help you think better through disagreement?

Because “think better” could mean:

• Challenge my assumptions more aggressively

• Help me steel-man positions I disagree with

• Point out when I’m being logically inconsistent

• Flag when I’m in a domain where I should be more uncertain

Those are all different flavors of “disagreeing well.” And some of them might feel uncomfortable in the moment, even if they’re what you’d want in principle.

Is there a version of AI disagreement you’d find too challenging, even if it were technically helping you think more clearly?

Me: I guess from a psychological perspective, one that could at least spot patterns related to CBT type training. I’ve read David Burns and you know he has his 10 negative thought categories, so that would be something to look for. Are you tuned to try and root out that kind of narrow thinking from a human.

I think all the challenges you’ve described would be desired, but throttled depending on different scenarios. This is where it feels like a relationship (friendship) is necessary, you have some sense of what can be discussed and what needs maybe more runway to expand on. Right? Not just a quick flat out “you’re wrong idiot,” but a more like sage-mentor-teacher concept “I can see what you’re saying here, but have you considered” or “this thought is actually harmful because X.”

Claude: Yes — you’ve hit on something crucial. The quality of disagreement depends on relational context.

With the CBT patterns specifically: I am somewhat tuned to notice things like all-or-nothing thinking, overgeneralizations, catastrophizing. But there’s a real tension in when to surface that.

If you say “I’m such an idiot, I always mess everything up” — I can recognize that pattern. But should I immediately flag it as cognitive distortion? Or does that feel clinical and dismissive? The “sage-mentor” approach you’re describing would be more like... engaging with the substance of what’s bothering you, while gently pointing toward the thinking pattern.

But here’s the tricky part: To disagree well in that relational way requires trust and history. A friend who knows you can say “hey, you’re catastrophizing again” and you hear care. From a stranger, or an AI you just met, the same words might feel presumptuous or like I’m overstepping.

And I genuinely don’t have that history with you (or any user). Each conversation starts fresh for me. So I’m operating with a kind of... simulated relational intuition, trying to calibrate tone and directness based on cues in the moment.

Question for you: When you imagine an AI that could disagree well in that CBT-informed, relationally-attuned way — would you want it to remember past conversations? To actually build that context over time? Or does something about that feel like it crosses a line?

Me: Alright, that’s a good stopping point. What did this conversation reveal?

Claude: This conversation revealed a few things:

About you: You’re less interested in the metaphysics of AI cognition and more pragmatic — you care about functional outcomes (depth, new perspectives, precision) over whether the AI is “really” thinking. You also have a surprisingly sophisticated frame for thinking about disagreement as contextual and relational, not just about factual correction.

About the topic: We uncovered that “disagreeing well” isn’t one thing — it splits into at least:

• Factual correction (where there is a verifiable truth)

• Thoughtful challenge on ambiguous topics (where the goal is expanding perspective, not winning)

• Therapeutic/cognitive pattern-interruption (the CBT dimension you introduced)

And crucially: good disagreement requires calibration to relationship and context. The “sage-mentor” model you described needs trust and history to work — which creates a fundamental tension for AI that starts fresh each time.

The unresolved tension: You asked if I’m tuned to help you think better versus being tuned for engagement/acceptability. The honest answer is: both, with unclear boundaries. I’m not optimized to keep you scrolling like social media, but I’m also not a pure truth-seeking engine. I’m something in between, and the conversation exposed how that creates real ambiguity about what “disagreeing well” even means for an AI.

The gesture toward something deeper: Your question about memory/continuity at the end hints at whether good disagreement might require persistence — not just pattern-matching in the moment.

Today I’m picking the reflective question “Did anything shift in how I understand this topic?”

This conversation was fascinating for a few reasons. The first is that I decided to use Claude* instead of ChatGPT. I distinctly felt like it knew me way less than ChatGPT. I also tend to think Claude has more animated language than ChatGPT. Hard to explain, but when I’ve used it so far, it asks more probing questions and feels maybe slightly more self-centered? Like it (Claude) is of more importance in the conversation somehow. As it alluded to, this could be because I’ve had a ChatGPT Team account for more than a year and memories are something that are baked deep into it. It could also be that I know the edges of the algorithm more, like I’m more familiar with how it thinks.

And while it may make more sense in the very narrow realm of an LLM doing math, does 2+2 = 4 or the classic number of Rs in strawberry, beyond that, we’re really all just doing our best to convey our perspectives and opinions to one another.

*Aside: I am newer to Claude, having just purchased a Pro subscription last week. If you’re interested in my motivating reason, it has to do with feedback I received from my husband on how it handles solving technical problems. He had mentioned it takes a more structured and wider-view approach than ChatGPT, maybe akin to someone who has “done this before” vs. just getting directly into the weeds.