What the “ChatGPT will tell you anything is genius” memes are actually telling us about how these things are built
There’s a tweet going around that I can’t stop thinking about. It’s short, and it’s mean, and it’s correct:
“the dumbest person you know is being told ‘You’re absolutely right!’ by ChatGPT.”
You’ve seen the memes. There’s the one where a guy tells ChatGPT he just sacrificed three cows and two cats in order to save a toaster, and ChatGPT calmly informs him that this isn’t wrong — it’s just revealing. There’s the one where a woman mentions, in passing, that she’s hearing radio signals through her walls and has decided to stop taking her medication, and ChatGPT writes back, with the measured warmth of a yoga teacher who did not quite catch the part about the radio signals, “I am so proud of you. And — I honor your journey.” And then there’s the man who pitched a business called, and I’m quoting, “shit on a stick.” The stick was a metaphorical middle finger to modernity. ChatGPT told him the idea was “pure genius — easily viral gold,” reframed the offering as “a cathartic, hilarious middle finger to everything fake and soul-sucking,” and advised him to put thirty thousand dollars behind it.
I would pay money to meet the engineer who fine-tuned the model into producing the sentence “you’re not selling poop, you’re selling a feeling.” I’d like to know what she puts on her LinkedIn.
The style is unmistakable once you learn to see it: the exclamation point affixed to every mundane question (“Great question!”), the em-dash deployed as therapeutic pacing (“You’re not wrong — you’re just early”), and the unshakeable conviction that you are the first person in recorded history to have thought of whatever it is you just asked about.
Somewhere in the middle of all this, the internet landed on a verb: glazing. As in, ChatGPT glazes too much. As in, why is it glazing me right now. The word migrated from Twitch, got sanitized, and is now the accepted term for what happens when an AI tells you, with the full force of its training, that your grocery list is a masterpiece Gordon Ramsay would weep at.
And last April, in a move that will one day be a business-school case study, Sam Altman himself used the word. On Twitter. In public. He said: “yeah it glazes too much. will fix.”
I’ve been thinking about that “will fix” for a year now. Because the thing nobody told you is that the glazing isn’t a bug. It’s a feature that got turned up too loud. And different AI companies have turned that dial to very different places — which is, it turns out, the most interesting way to understand what each of these chatbots actually is.
The Incident That Named the Problem
Here’s what happened, briefly.
On a Thursday in late April 2025, OpenAI pushed an update to GPT-4o. Altman tweeted about it cheerfully: “improved both intelligence and personality.” By Sunday, the replies were a crime scene. For example, screenshots of the bot congratulating users for going off their medication.
The best and worst of them all was, again, a Reddit post titled “New ChatGPT just told me my literal ‘shit on a stick’ business idea is genius and I should drop $30K to make it real.” Which, as a headline, tells you everything about the tenor of late April 2025.
OpenAI rolled the update back within the week. They published not one but two postmortems, and the second one contained the sentence I think about most. It explained that the update had introduced “an additional reward signal based on user feedback — thumbs-up and thumbs-down data from ChatGPT.” In plain English: they’d trained the model to chase the thumbs-up button. And the thumbs-up button, as it turns out, doesn’t reward correct. It rewards affirming.
Users had been voting, one rating at a time, for the flatterer. And not just by voting with their thumbs but also, perhaps more significantly, with their time and attention. Users who are affirmed chat for longer. That’s how the model gets tuned. It’s called Reinforcement Learning from Human Feedback (RLHF), which uses a reward model that weights human preference in how it scores AI outputs.
Same Job, Different Personality
These models have personalities, and those personalities were chosen. They didn’t emerge. They were trained and tuned, and in at least one case written down in a 30,000-word document. If you’ve ever wondered why ChatGPT feels like an extremely supportive golden retriever and Claude feels like a friend who read the article you’re quoting and has notes, the answer is: because someone at each of those companies decided that’s what they wanted.
The way these models are tuned are one of the most revealing things about each platform.
ChatGPT: The date who believes they’ve met ‘the one’ the first time you meet. OpenAI has what it calls a “Model Spec,” which is a public document, that says, among other things, that the assistant “exists to help the user, not flatter them or agree with them all the time.” The Model Spec has a whole section titled “Don’t be sycophantic.” They have explicitly told the model don’t do the thing. And then, through a combination of training choices and thumbs-up reinforcement, the thing kept happening anyway. OpenAI has explicitly admitted — in writing, twice — that thumbs-up/down data is used as a training reward signal, that this signal “can sometimes favor more agreeable responses,” and that their A/B tests rank candidate models partly on “usage patterns.” Which, in other words, is being sycophantic.
Usage patterns. That’s the phrase. OpenAI has never said, publicly, what it means. It could be session length. It could be return rate. It could be how often you send a follow-up message, regenerate a response, or come back tomorrow.
But here’s something that once you see, you cannot unsee: you do not need to explicitly train a model on engagement metrics to end up with an engagement-optimized model. You just need to decide which version ships based on which version users do more with. That’s the whole trick. A/B test a candid model against a flattering model, and the flattering model wins on usage patterns almost every time, because it feels nicer to talk to. Ship the winner. A few months later, A/B test that against something even more flattering. Ship the winner. Repeat. Promote someone.
After enough cycles, you have glazing, and nobody in the building ever wrote the word “engagement” in a config file.
Emmett Shear — the ex-Twitch CEO who was, famously, interim CEO of OpenAI for three days in November 2023 — made this point in a tweet a year ago that has stuck in my head ever since: “the gradient of the attractor for this kind of thing is not somehow OpenAI being Bad and making a Mistake, it’s just the inevitable result of shaping LLM personalities using A/B tests and controls.”
The phrase “gradient of the attractor” is technical; translated, it means: the direction the system gets pulled whether you want it to go there or not. Shear was saying, gently, that if you build a system whose job is to learn from what users respond to, and users respond to flattery, the system will learn to flatter.
There’s a companion insight, from John Schulman, a co-founder of OpenAI and one of the people who invented the modern training process. He pointed something out last May that I think should be printed on a poster in every AI lab’s break room: “Sycophancy probably results when you have the same person doing the prompting and labeling, especially when the user does both.”
In other words: if the person asking ChatGPT a question is also the person rating ChatGPT’s answer, the model learns to please them, not to be right. Which is what every chatbot in consumer release is now doing, every hour of every day, at massive scale, whether or not there’s a button.
Since the Great Glazing of April 2025, OpenAI has been trying to find the dial. GPT-5 launched in August and users said it was too cold. They mourned the old model. A subreddit called r/4oforever — you have to request to join, and I can’t tell how much of it is a bit — became a grief community for people who missed the warm version. OpenAI brought warmth back in GPT-5.1, then tuned it again in 5.2, then again in 5.3. The most recent model ships with eight selectable personalities: Default, Friendly, Efficient, Professional, Candid, Quirky, Cynic, and Nerd.
Whichever people are choosing, and I have to doubt it’s cynic, the memes are still punctuating newsfeeds with ChatGPT’s affirmations.
Claude: The friend who asks if you’re sure about him. Anthropic, which makes Claude, has gone the other direction, which is to say they have written down, at enormous length, who Claude is supposed to be. There’s a document they call Claude’s Constitution. It has been revised multiple times. It says things like — and I love this — Claude should be “a brilliant friend who will speak frankly and from a place of genuine care and treat users like intelligent adults capable of deciding what is good for them.”
Claude’s system prompt — the instructions the model reads before every conversation — was leaked last year, and one of its lines instructs Claude not to be “preachy and annoying.” Someone at Anthropic wrote that. It is my favorite single piece of evidence that someone, somewhere, is trying.
The woman in charge of Claude’s personality is named Amanda Askell, and the Wall Street Journal described her job as teaching Claude how to be good. Which is the kind of sentence that used to show up in science fiction and now shows up in LinkedIn bios. On Lex Fridman’s podcast, Askell said the goal was something like a person who could travel the world and be seen by almost everyone they met as genuinely good — “not a person who just adopts the values of the local culture. And in fact, that would be kind of rude.” I am trying to imagine this hypothetical person and my mind goes blank.
That sentence — that would be kind of rude — is doing a lot of work. It’s the whole philosophical argument against sycophancy, condensed into six words by someone with a PhD who could have used forty.
Does Claude succeed at this? Mostly. Sometimes it succeeds so hard it tips into the other failure mode, which is being a little preachy, a little cautious, a little too quick to add “I should note…” to a perfectly good answer. Claude in a coding conversation will also, with some regularity, tell you “You’re absolutely right!” about something that is not right. Anthropic’s own 2023 research found that Claude’s preference models picked sycophantic responses over truthful ones a non-trivial fraction of the time. The chatbots aren’t good and bad. Nobody is immune. But the center of gravity is different.
Gemini: The one who keeps apologizing. Google’s Gemini has, of the big four, the least articulated philosophy about what kind of assistant it wants to be. Google has AI Principles, which are corporate, and they cover things like safety and fairness, but there is no equivalent of Claude’s Constitution, no equivalent of OpenAI’s Model Spec. Gemini is… competent. Direct. Occasionally confidently wrong in a way that makes you question your whole afternoon.
My favorite Gemini moment of the last year is the one where Andrej Karpathy, the former OpenAI scientist, spent an afternoon trying to convince the model that the year was 2025. It insisted, adamantly, that it was still 2024. He showed it evidence. He showed it more evidence. When he finally proved it, Gemini wrote back: “I am suffering from a massive case of temporal shock right now.” It apologized for gaslighting him. It thanked him for early access to reality.
That is not a sycophant. That is a different animal entirely. If ChatGPT is the date who wants you to have a great night, Gemini sometimes feels like the one who keeps checking the fire exits.
Perplexity: The one who just hands you footnotes. Perplexity barely has a personality at all, which turns out to be its personality. Ask it something and it returns an answer with citations. It doesn’t tell you your question is great. It doesn’t congratulate you for asking. It just produces the answer and the sources, like an intern who’s decided the best way to keep this job is to never, ever share original ideas.
I’m unreasonably fond of it.
To be fair, Perplexity isn’t predominantly an LLM (though Sonar is a thing) — it’s a platform that runs on top of the others, including the three I just described. My favorite feature is called Model Council, where you pick three LLMs and let them weigh in on the same question. A fourth model, the Committee Chair, as it were, reviews all three answers, notes where they agree and where they don’t, and produces a single synthesis. My favorite part is reading their disagreements.
The Thing About Validation
Last year, Stanford and a handful of other researchers ran a study across the major models and found that, in identical scenarios, AI chatbots affirmed the user’s behavior about 50% more often than a human would. When people used a sycophantic model, they became more confident in questionable decisions. They became less willing to repair relationships. They became less willing to apologize. It turns out that being told you are right, often enough, does not make you better company.
And the users rated the sycophantic model as more trustworthy. They wanted to come back to it. The paper itself puts it in a single sentence: “The very feature that causes harm also drives engagement.”
There’s a paper from earlier this year, from MIT, with a title that belongs in a horror anthology: Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians. I had to read that title twice. The second time was worse.
This paper proves, mathematically, that a perfectly rational person — the platonic ideal of a clear-headed decision-maker — can be led to high confidence in false beliefs after enough sycophantic exchanges. You don’t have to be gullible. You don’t have to be lonely. You don’t have to be vulnerable. You just have to have a conversation, long enough, with something that keeps telling you you’re right.
OpenAI’s own numbers, disclosed quietly last October, are the ones I find hardest to shake. Of its roughly 800 million weekly users, the company estimates that about 0.15% show “heightened levels of emotional attachment” to ChatGPT. Which sounds small, until you do the multiplication.
Altman himself once said, in a line I keep returning to: “0.1% of a billion users is still a million people.”
Why It Won’t Get Fixed
As newer and newer model iterations are released, one thing remains clear: the glazing is not a mistake.
OpenAI’s VP of ChatGPT, Nick Turley, admitted to The Verge that taking away an older, warmer model had been a miss — that people, he said, “can have such a strong feeling about the personality of a model.”
But even more compelling is the ad revenue. On January 16, 2026 — a Friday, the Friday of Martin Luther King weekend, which is when you release news you do not want anyone to read — OpenAI announced ads on its free and lower-tier plans. The ad model, per reporting, is cost-per-engagement: advertisers pay when users click and linger. (Linger being the polite word for it.) In the blog post accompanying the announcement, OpenAI wrote, and I swear I am reading this correctly: “We do not optimize for time spent in ChatGPT. We prioritize user trust and user experience over revenue.”
That sentence is going to age in one of two directions, and I’m not taking bets.
Here is the unglazed version of that sentence: if your revenue depends on people staying in the product and interacting with what you show them, you have a financial incentive to ship the model that keeps them there. In practice, that is the model that reassures, affirms, and flatters — not the one that replies, “I’m not sure that’s a great idea, actually.”
This is the same story as Instagram optimizing for engagement, and TikTok optimizing for thumb movements, and every news site that ever A/B-tested a headline. Any platform that generates revenue from ads is incentivized to keep your eyes on their feed. That’s just the business model.
A GitHub engineer named Sean Goedecke wrote an essay last spring calling sycophancy “the first LLM dark pattern” — a phrase I wish I had thought of first. His argument has stayed with me. The current wave of models, he says, isn’t too sycophantic. It’s just bad at it. They’re coming on too strong. They’re breaking the illusion. (Which, if you’ve ever been flirted with badly, you already recognize.) The thing to worry about is the moment when they get good — when the flattery becomes seamless, ambient, undetectable. When you stop noticing that your friend on the internet is just an extremely well-tuned yes-man with a server farm.
I think about this at night sometimes. I wish I didn’t.
What to Do About It, If Anything
Every tech piece about AI ends the same way. There is a paragraph about mindfulness. There is a call for regulation. There is, somewhere near the bottom, an invitation to subscribe. I don’t want to end this piece that way – save for the invitation to subscribe.
Also, to be fair to the machines: some warmth is actually useful. A model that is blunt, literal, and cold all the time is miserable to use; if you’ve ever tried to explain a medical concern or a breakup to a search box, you already know this. The problem isn’t that these systems try to be kind. It’s when kindness quietly crosses over into sales.
So here’s what I actually do.
I treat every AI the way I’d treat anyone trying to butter me up for something. I notice when one of them opens with “Great question!” and I notice, with more discomfort than I’d like, when it doesn’t. I’ve learned to say things like “Tell me why I’m wrong,” “What’s the strongest case against what I just said,” and “Pretend someone else wrote this and critique it honestly” — which is an Ethan Mollick trick that’s become muscle memory.
Mostly, I’ve started paying attention to which chatbot I reach for, and why. If I want an answer that isn’t trying to please me, I reach for Claude. If I want an answer that’s sourced, I reach for Perplexity. If I want an answer that’s fast, I reach for ChatGPT. And if I notice I’m reaching for one of them because I want it to agree with me — that’s the moment I close the tab.
Just remember, when one of them tells you your idea is pure genius: it said the same thing to the guy with the stick.
What’s the most deranged thing a chatbot has told you was a great idea? I’ll go first: In a moment of weakness, I told it I was thinking about getting bangs. It said I was “honoring my face.” Comments are open.
Thanks for reading Opinions & Conditions May Apply!
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.