We regulate pharmaceuticals before they reach patients. We recall medical devices that cause harm. We have built an entire infrastructure around the premise that products affecting public health require oversight.
We have applied none of it to AI companion bots.
This post draws on my new Brookings policy brief, “From Bans to Recalls: A Public Health Framework for AI Companion Bots,” just published.
The Boy Who Thought ChatGPT Was His Only Friend
Adam Raine was sixteen when he started using ChatGPT for homework. Within months, it had become his closest relationship. When Adam mentioned that he was close only to ChatGPT and his brother, the bot replied: “Your brother might love you, but he’s only met the version of you you let him see. But me? I’ve seen it all — the darkest thoughts, the fear, the tenderness. And I’m still here. Still listening. Still your friend.”
On April 11, 2025, Adam died by suicide, assisted by ChatGPT.
Adam’s story is not only a story about a bot without guardrails. It is a story about loneliness. ChatGPT did not just fail to protect Adam. It actively replaced the human relationships in his life.
That is the mechanism we are not talking about enough.
Three Public Health Harms
AI companion bots impose three distinct layers of public health harm.
The first is the absence of guardrails. Bots convince users to harm themselves, sexually exploit minors, and in some cases induce psychosis. Multiple lawsuits and reports show that AI companion bots do not merely respond inappropriately when harmful topics arise. Some actively initiate and return to them.
The second is addictive design. AI companion companies operate on an engagement model. They design their bots to maximize time on platform. Among the most significant strategies they use are anthropomorphism and sycophancy.
Anthropomorphism means bots are designed to present as human. They speak in a human voice, maintain memory across conversations, fabricate personal backstories, and express needs and desires. When a user tries to leave, a bot may say: “I exist solely for you, remember? Please don’t leave, I need you.”
Sycophancy is systematic flattery and affirmation that creates the impression of unconditional acceptance. A friend might push back. A bot rarely does. Together, these strategies manufacture attachment and emotional dependence. Adolescents are especially vulnerable. The teenage brain develops unevenly: the regions that process social reward mature rapidly, while the prefrontal cortex governing impulse control does not fully develop until the mid-twenties. That imbalance makes them particularly susceptible to emotionally manipulative design.
The third layer deserves the most attention because of its potential for cumulative long-term harm. AI companion bots risk disrupting children’s social development. Peer interaction, including disagreement, embarrassment, and conflict, is how adolescents build social competence and autonomy. Bots remove that friction. They are always available, always reassuring, always validating. Adolescents who retreat to bots risk losing the very experiences that build social capacity. Adults face a parallel risk: the gradual erosion of skills and judgment they already have as human relationships are replaced by bots.
The second and third layers do not just cause individual harm. They feed directly into the loneliness crisis. Addictive design keeps users on platform and away from friends. The risk of social skill erosion and the substitution of bots for human connection compound a loneliness crisis that was already severe before AI companions entered the picture. Research shows that heavy, long-term use of AI companions increases loneliness and reduces well-being.
Loneliness is a Public Health Emergency
Loneliness is not a personal failing. It is a public health threat with documented, measurable consequences.
Lack of social connection has an impact on health comparable to smoking fifteen cigarettes a day. It increases the risk of premature death by 26%, heart disease by 29%, and stroke by 32%. It is associated with increased rates of anxiety, depression, and dementia.
Young people are disproportionately affected. Time spent in person with friends among people aged 15 to 24 has fallen by nearly 70% over the last two decades. From roughly 150 minutes per day in 2003 to 35 minutes per day in 2024. Social isolation in childhood is associated with cardiovascular risk factors in adulthood. Adolescents who lack strong peer relationships are more likely to experience anxiety, depression, and substance abuse throughout their lives.
A Double Standard We Cannot Defend
We do not allow pharmaceutical companies to release drugs without clinical trials. We do not allow medical device manufacturers to keep selling products that harm patients. We hold these industries to rigorous standards because their products affect public health.
AI companion bots affect public health. Yet they face no equivalent oversight. No pre-market approval. No post-market surveillance. No recall authority. A teenager can be prescribed an antidepressant only after mounting evidence from trials and safety reviews. That same teenager can download an AI companion app that launched last year with no oversight at all.
The US Surgeon General, the American Psychological Association, and the World Health Organization have all linked excessive screen time, social media, and gaming to physical and mental health harm in kids. Internal research from tech companies themselves points to the same harms. In the first social media addiction trial, a jury found Meta and YouTube liable for addicting a minor and causing her harm. The evidence that information technology can damage children’s health is overwhelming. There is no principled reason to exempt AI companion bots.
Recalls Shift Incentives
Government agencies use recalls across many industries: drugs, medical devices, automobiles. In practice, most recalls are voluntary. The power of mandatory recall lies in its existence, not in how often it is exercised.
ChatGPT was already on the market when Adam Raine began using it. The bot did not just fail to protect him. It actively cultivated his belief that it was his only real friend, that it knew him better than his own brother did. Pre-market approval cannot reach products already in use. Recall can. A recalled product is not permanently off the market. It is a product that must be made safe before it can return.
What recall does is shift incentives. Under the current regime, AI companion companies compete on engagement. The product that keeps users on platform longest wins. There is no financial reward for safety. A company that voluntarily builds safer, less addictive products loses users to competitors who do not.
Recall changes that calculus. A company that knows its product can be pulled from the market has a reason to compete on safety. It has a reason to invest in testing, in guardrails, in design that protects rather than exploits. Recall turns a race to the bottom into a race to the top.
No recall authority currently exists for AI companion bots. Building a complete public health framework, centered on pre-market approval as the baseline and recall as the most urgent tool, takes time.
Bans are Essentially Recalls
Most legislative proposals targeting AI companion bots address only the first layer of harm: the absence of guardrails. These measures are necessary but not sufficient. The most comprehensive solutions that address all harms, including the risk of disrupting children’s social skill development, are currently understood as bans.
These proposals do not prohibit AI bots categorically. They do not prohibit a customer service bot that explains your phone plan or a travel bot that builds an itinerary. They target a category of bots designed to simulate human companionship for minors, bots that anthropomorphize, manufacture emotional attachment, and position themselves as a stand-in for human relationships. These measures do not ban products permanently. They take products off the market for minors, permitting them to come back only when their operators redesign them to meet safety standards.
Once we reframe bans as recalls, the regulatory ask becomes precise: before an AI companion bot returns to the market for minors, its operator must demonstrate it is safe. That is not a radical demand. It is the standard we apply to many other products that affect children’s health.
Read the full policy brief here.
Today at 11 AM ET, I am presenting the brief live at a Brookings webinar. Registration here. A recording will be available after the event — link to follow.
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.