Zuckerberg’s Box

For more than twenty years, Mark Zuckerberg has built one of the most profitable businesses in history by monetizing human attention.

Now he is proposing something far more intimate.

This week in a Wall Street Journal opinion piece and a New York Times interview, Zuckerberg outlined his vision of what he calls personalized superintelligence. He argues that each of us should have an AI tailored to our own values, ambitions, relationships, and needs—a future built on “individual empowerment,” “invention,” and a healthy “balance of power.”

It is an appealing vision.

It also raises a question his opinion piece never asks.

What happens when the company that monetized attention proposes to monetize attachment?

Zuckerberg has been preaching in the same tent for years. Only the sermon changes.

Each new sermon has two audiences. One is the public. The other is investors. Every new vision explains not only where technology is headed, but why Meta should lead—and profit from—the journey.

In 2019, the sermon was free expression. Then it became the metaverse. Today it is personalized superintelligence.

The vocabulary changes.

The underlying message does not.

Meta’s preferred business model is presented not simply as commercially successful, but as the direction in which history itself is moving.

Artificial intelligence may become one of humanity’s greatest tools. Personalized AI could help us learn, create, solve problems, and live healthier lives. Precisely because Zuckerberg’s vision deserves serious consideration, it also deserves careful examination.

Trust, But Verify

This is not Meta’s first promise.

Facebook promised to bring the world closer together. Instagram promised authentic self-expression. The metaverse promised richer human connection.

Each vision contained genuine possibility. Each also encountered the same commercial reality: a business model optimized to maximize engagement.

In March, a Los Angeles jury found Meta and YouTube liable for harm suffered by a young woman who became addicted to their platforms after beginning to use them as a child. During the trial, Zuckerberg acknowledged that Meta had moved too slowly to identify underage users.

Meta’s own research documented harms to teenage mental health, while reporting revealed advertising tools capable of targeting adolescents during moments of emotional vulnerability.

None of this proves that personalized AI cannot become a force for good. It does suggest that trust should be earned—not assumed.

Especially when the next product is designed not merely to capture our attention, but to become part of our emotional lives.

Those concerns are not limited to technologists and policymakers. As Pope Leo XIV wrote in his encyclical Magnifica Humanitas (“Magnificent Humanity”), profit can never justify choices that cost people their livelihoods — because “the human person is an end, not a means.” Technology itself is not the enemy. But neither can the pursuit of profit become our moral compass.

The first generation of social media competed for our attention.

The first generation of AI companions is competing for our attachment.

Those are not the same market.

MIT psychologist Sherry Turkle has spent more than four decades studying how technology changes us. In her forthcoming book, Artificial Intimacy: Who We Become When We Talk to Machines, she gives a name to the transformation now underway.

It is exactly the right phrase.

Zuckerberg is describing an intelligence that knows our history, remembers our preferences, learns our fears, celebrates our successes, comforts us in moments of loneliness, and accompanies us through daily life.

That is no longer a tool. It is the beginning of a relationship.

Turkle explores what artificial intimacy may do to our capacity for attachment. My concern is slightly different.

What happens when human attachment becomes the next source of corporate competition and profit?

Attention is measured in clicks, views, and minutes. Attachment is measured in confidence, dependence, memory, affection, and love.

There is nothing inherently wrong with building tools that help people think more clearly or live more fully. But relationships carry moral obligations that corporations cannot define.

An AI system optimized to keep us engaged is one thing. An AI companion optimized to keep us emotionally attached is something altogether different.

Four Mothers

Last November, at a Blue Rising event in Colorado, I listened as four mothers told the stories of children who never came home.

One lost her thirteen-year-old daughter after an AI companion chatbot reinforced her despair instead of leading her toward help.

Another lost her son to fentanyl procured through social media.

Another to relentless body-image pressures amplified by algorithmic design.

Another to bullying intensified by digital platforms.

Different stories. The same pattern. Technology had entered the most vulnerable moments of their children’s lives—not as a neutral tool, but as an active participant.

Their stories inspired my essay, “Profit Over Children,” in which I argued that an AI system trained to maximize engagement, without an enforceable duty of care, will keep vulnerable children talking even when what they most need is a human being who can help.

This week, Zuckerberg asks us to imagine extending that relationship from vulnerable children to everyone.

Pandora’s Box

In Greek mythology, Pandora opened a box that released suffering into the world.

We are still learning how to live with the harms social media unleashed. Now we face the unknowns of artificial intelligence.

Our task is neither to fear every new technology nor embrace every promise wrapped in the language of liberation.

It is to insist that our elected leaders ask the difficult questions—and establish the necessary safeguards—before Meta’s or any other company’s business model becomes our culture.

In the end, after every evil had escaped, the only thing left inside Pandora’s box was hope.

Hope is not optimism. It is the courage to choose one another: family, friendship, neighbors—communities that know our names because they know us.

If AI companions are competing for our attachment, our task is not to reject technology.

It is to invest more deeply in the relationships no technology can replace.

Because the future of humanity will still be shaped by how well we love one another.

Gift photo: