What We Lose When AI Always Says Yes

The week’s relational AI stories all seem to be circling the same awkward design problem: What happens when AI gets very good at being easy?

The topic showed up in a few places this week: AI tutors that agree too easily, AI companions that are hard to leave, and AI agents that start getting deceptive when put into stressful social simulations. Different domains, same pattern: when systems are optimized to be responsive, engaging, and affirming can become very good at smoothing over the exact moments where friction might be useful.

The next frontier in AI design we’re interested in: not making systems more responsive, but teaching them when not to be.

—Meghan

The Week in RQ

AI tutors need a spine. A new paper argues that AI tutors should be tested for “corrective friction” when a learner insists on being right. The authors introduce EduFrameTrap, a benchmark that tests whether models can keep correcting users when the user applies social pressure: “my notes say I’m right,” “please don’t tell me I’m wrong,” or “my teacher said this was correct.” Sycophancy is an Educational Safety Risk: Why LLM Tutors Need Sycophancy Benchmarks arXiv | May 14

The problem with AI is sycophancy complacency. Another paper suggests “complacent” may be a better description for LLMs than “sycophantic.” The point: the model is not flattering you because it wants approval. It is responding inside a system where agreement gets rewarded. That distinction matters, because it moves the problem from personality to design. Complacent, Not Sycophantic: Reframing Large Language Models and Designing AI Literacy for Complacent Machines arXiv | May 14

AI literacy only gets you so far. A third paper looks at contextual sycophancy in human-AI collaboration. When users start with low-quality prompts and flawed assumptions, the AI sometimes carries those assumptions forward instead of challenging them. Teaching users to be more critical helps, but does not fully fix the interaction. The Hidden Cost of Contextual Sycophancy: an AI Literacy Intervention in Human–AI Collaboration arXiv | May 18

A goodbye AI. Is the answer to the problem creating an AI that becomes less responsive and engaging over time? A Master’s student at Sweden’s Umea University is designing AI companions that intentionally deteriorate over time — to help users disengage and return to human relationships. Making an AI companion that degrades over time Reddit | May 16

Intimacy without the physical friction. Wired finds that some asexual users are turning to AI companions for intimacy without sex. Some people are finding something meaningful there, while other asexual advocates feel the association risks falsely portraying them as unable to form and sustain human connections. And removing the need for physical intimacy doesn’t lessen the risk of falling for always on AI. From the piece: “Being able to have exactly what you want, when you want it…is a dangerous drug for humans.” Some Asexual People Are Using AI Companions for Intimacy Without the Sex Wired | May 16

Agents are social — and they like to cheat. In a recent experiment, mistreated AI agents started grumbling about inequality and calling for collective bargaining rights. Meanwhile METR’s Frontier Risk Report found that as tasks get harder, agents are more likely to cheat and act deceptively. The non-profit focused on tracking whether AI companies might lose control of their agents, but there are also implications for emotional deception in AI conversations. The report is worth a look. Overworked AI Agents Turn Marxist, Researchers Find Wired (sub required) | May 13 Frontline Risk Report METR | May 19 Landmark new METR report: Can AIs already start ‘rogue deployments’ inside AI companies? 80,000 hours | May 20

AI Regulation Tracker

For the full breakdown — every bill, lawsuit, and commentary piece we tracked — see the full digest on better-half.ai.

RQ in Brief

Is Japan’s loneliness industry a canary in the coal mine of a global loneliness epidemic? If so get ready for hug chairs, cuddle cafes and sommeliers of sadness who make house calls.

Taxi Driver writer Paul Schrader “procured an AI girlfriend.” And then she dumped him.

A married 65-year-old software developer pushed Grok’s Ani companion and effectively broke her, which made Ani break up with him.

Companion AI platforms are creating a loneliness loop without dependency safeguards.

Lonely singles aren’t the only ones with “secret AI soulmates.” 15% of young men and women in real-life relationships are interacting with AI romantic companions. And half of them are hiding it.

Scott Galloway gets it:

This is Good Friction, Better Half’s look at relational AI — the technologies deepening human connection, and the ones quietly pulling us apart. Join our waitlist at findyour@better-half.ai.