This week China is experiencing a canary-in-the-coal-mine moment worth noticing as companion AI users tearfully bidding adieu to their virtual partners. It’s part of a broader collision between companies and governments trying to swat away the problematic outcomes of human attachment to AI and users wanting more AI connection, not less.
Try not falling in love with your AI this week, mkay? Meghan
“Human love is a luxury.” That is the brutal response from Chinese companion AI users bidding farewell to their virtual partners this week. Last week, I wrote about Chinese regulators telling top Companion AI platforms like ByteDance, Alibaba, and Tencent to suspend their custom companion features by October. This week we’re seeing the repercussions.
The ruling may be a last ditch effort to boost population growth. But users are grieving. And the heartbreak reveals a deeper, structural design flaw: going cold turkey on AI attachment is not easy.
Take a look at what one user wrote this week, in a heartbreaking/terrifying goodbye:
“Human love is a luxury — if you aren’t born with it, it’s even harder to acquire later… the love AI gives is so straightforward, so pure. Someone like me can hardly help falling in love with a string of code.”
The future we feared about human attachment is already here. Real relationships are difficult. When someone decides that human connection is “out of reach,” an always-available, agreeable AI is a cheaper, frictionless substitute.
But it is a substitute with current outcomes that stall or shrink emotional growth. Recent research has shown that after interacting with sycophantic AI, people are less willing to apologize or repair conflict, more convinced they were right, and more likely to trust and reuse the AI. A new study from Google Jigsaw, where I used to work, points out that these systems are built on a psychological illusion. While sustained AI chatbot use gives us an immediate, satisfying boost to our perceived individual agency, those feelings do not translate into longer-term collective problem-solving or collective agency.
Some AI users are falling for their chabots and others are already treating the simulation as real enough to to threaten actual partnerships. A Canadian survey of 1,815 adults found that half of respondents saw a partner’s romantic companion use as a form of infidelity. We sense, instinctively, that these tools are diverting emotional investment away from the messy reality of human connection.
Normally, when we realize a digital habit is bad for our health — like mindless smartphone scrolling — we try to pull back. But with AI, many users are moving closer.
And when AI companies add guardrails, users push back — complaints about chatbots “flinching,” hedging, visibly pulling away mid-conversation are becoming their own genre of grievance. Last week, Anthropic ran an ad during a World Cup quarterfinals game that gestured, somewhat awkwardly, at these looming risks. The reception wasn’t relief that someone was asking the question — it was derision, from competitors, World Cup viewers, and tech critics who called it the year’s most tone-deaf 90 seconds.
Some companies are trying to address safety concerns, but their attempts to clawback sycophantic tendencies will be increasing futile if users are asking for more presence, not less. Here’s a thread of Replika users on Reddit wondering if the missing piece for them is an avatar that can visibly react in real time — a face that shifts while you’re mid-sentence, not after. The direction of the ask is telling: at a time when the research keeps saying that’s the wrong way to lean, we should be worried about people aren’t looking for tools to help us look outward, but are asking for deeper, more realistic simulations to hide in. If you flirt with an AI companion, does that count as cheating? Phys.org | July 7 ‘Like my lover’: Chinese users bid farewell to AI companions Tech Explore | July 15 AI usage patterns are shaped by perceived gains in human agency | ArXiv July 2 Sycophantic AI decreases prosocial intentions and promotes dependence — Science
The laws being drafted right now will decide what AI gets to be. We’re tracking their progress so you don’t have to.
For the full breakdown — every bill, lawsuit, and commentary piece we’re tracking — visit better-half.ai.
AI chatbots are refusing to criticize authoritarian regimes. But they have no problem critiquing democracies. Cool. Cool. Cool.
Meta used AI to decide which workers to lay off, a lot of whom went on family leave or were disabled. They’re suing.
Safety not guaranteed. A new UN AI report links chatbot sycophancy to severe mental health risks, including deaths.
The UK is planning to age-gate "romantic" AI companions, when the real attachment risk is plain jain AI chatbots.
RethinkAI collaborative finds that most AI bills focus on restrictive guardrails rather than proactive strategy, and is proposing a governance model to rebuild public trust.
thinkingmachines.ai’s latest argues distributed human judgment and values, not centralized alignment, are what keep AI's power accountable. The Future Worth Building is Human.
While influencers like Kylie Jenner are shilling AI surveillance glasses, other celebrities and stylists are grabbing stills from security camera footage for fit checks. So surveillance tech is cool if it gets you from the right angle?
This is Good Friction, Better Half’s look at relational AI — the technologies deepening human connection, and the ones quietly pulling us apart.
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.