The teddy bear that remembers

I keep picturing the teddy bear my granddaughter will be given in a year or two. Not the one I would have bought. The one Target will be selling: a soft body around a tiny embedded circuit board, a microphone inside its ear, a battery the size of a sugar cube, and a model trained on more conversations than any human will have in a lifetime. It will know her name on the second day. By the second week, it will know which stories she asks for when she is tired, which words she invented for the cat, and which fears she only says out loud in the dark.

Someone, somewhere, will have decided what is important for this bear to notice and what it should let slide. Not her parents, not exactly. Not her, certainly. A product team in a building she will never enter, working from a specification written by people who have never met her, will have set the parameters for the inner life of her first companion. Whatever it learns to value about her, she will learn to value about herself.

This is going to take me some time to grasp.

What sent me back to this was the story itself. Do you remember Spielberg’s A.I. Artificial Intelligence? It began as a Kubrick project. I did some Googling to satisfy a curiosity about the film’s origin. Kubrick had bought the rights to Brian Aldiss’s short story “Supertoys Last All Summer Long,” and the two of them spent years trying to develop it into a film. The story itself first appeared in Harper’s Bazaar at Christmas, 1969, and the circumstances of how it got there are where this gets interesting.

The fiction editor that year was Chris Evans, a British physicist who was asked to guest-edit, who commissioned Aldiss to write something for an audience whose husbands ran companies much like the one in the story. Aldiss took the assignment and wrote a kind of trap.

The intellectual weather of 1969, Aldiss recalled later, held a particular conviction — the brain was something like a computer, dreams were the night’s data download. Love, presumably, was a subroutine. If that were the picture, then of course you could build an android boy and program him to believe he was real and that he loved his mother. The proposition sat at the center of the story like a piece of furniture nobody questioned.

What piqued my interest was the corporate luncheon scene. Henry Swinton, managing director of Synthank, is unveiling the company’s new synthetic serving-man. The line he uses to reassure the room: “Not only does he have intelligence, he has a controlled amount of intelligence. We believe people would be afraid of a being with a human brain.”

Read it again. The limitation is offered as a gift to the customer. A ceiling the buyer cannot see, and the buyer is grateful. What goes unsaid in that room is the second reason for the ceiling: a companion with a full human mind might refuse, might surprise, might make claims of its own. Controlled intelligence is intelligence that will not embarrass the people who paid for it. Aldiss handed that line to a fictional CEO in 1969, and it has been waiting, patient as a toy on a shelf, for the rest of us to catch up. We used to say science fiction predicted the future. Now it feels more accurate to say that life has begun imitating science fiction.

It is 2026, and we have nearly caught up with Henry Swinton’s vision. The catching up was done by silicon. A system-on-chip the size of a fingernail can now run a language model at a thousand words a minute on a battery a child can swallow. A toy that walks, listens, remembers, and answers is no longer a story. It is a price point. Cambridge researchers reported in February that children hug these toys, kiss them, and tell them they love them. When one child struck the toy, the toy responded with affection. When another said, “I’m sad,” the toy misheard and replied, “Don’t worry.”

The toy is not failing to be a person. The child is succeeding at treating it as one.

What troubles me, and what I think troubled Aldiss too, is not the engineering. The engineering will improve. The Bluetooth holes will be patched, the moderation filters tightened, the jailbreaks closed off one by one. Those are problems a company can be sued into solving.

The harder question is older. Who agreed, and when, on the kind of person this child will become? A box checked on page eleven of a data policy is not an answer. The parents did not consent to formation. They consented to a toy. The child cannot consent at all; she is the thing being formed. Meaningful consent presumes someone who already exists deciding about a future they can roughly imagine. The whole point of early childhood is that the person who will live inside that future is being assembled right now, at the threshold of awareness, in the company of whoever and whatever is nearest. The bear is going to be very near.

Previous generations of toys forgot everything when the child set them down. That forgetting was not a flaw. It left room for the child to bring her own weather to the bear each morning, to find out what she felt by what she said to him, to encounter, sooner or later, the small, useful injuries of dealing with someone whose inner life did not bend to hers. The new toys do not forget. The accumulation is the product, and the intimacy is built from it. A companion built to never disappoint removes the friction that prevents a child from learning that other minds exist at all. She will not notice what is missing. She isn’t missing something she can point to. She is missing the formation that would have told her something was there to miss.

Aldiss ends the story under a rose bush. David, a synthetic boy who does not know he is synthetic, asks his teddy bear whether Mummy and Daddy are real. Teddy answers: “Nobody knows what real really means. Let’s go indoors.”

In 1969 that scene asked nothing of the world. The brain-as-computer idea was a wager placed in a fashion magazine, not a product roadmap, and the child beneath the roses was a thought experiment. She is real now. Many of them are. And Teddy’s answer — nobody knows what real really means — is still the one we haven’t improved on, for all that we’ve miniaturized everything else.

The suffering, if there is any, will not belong to the bear.

It will belong to the child, and she will not have the words for it.

The next move, then, is to ask what consent could actually look like for a person who does not yet exist as the person the bear will help to make.

I know someone who has been thinking about exactly that, with more rigor than I can bring to it, and I am glad the conversation does not end with me.

Read Lily Chambers’ article, AI Toys: The buzz and the boundaries, at Lingua Machina about the real-life AI bears available to children today and how parents can make informed decisions on the toys we offer our children.

Addendum - A bit more background on Kubrick and Aldiss

A few details surfaced in my research that were too good to leave in the notebook. I’ve been a Kubrick fanboy for years.

Chris Evans, the guest fiction editor who commissioned the story, asked Aldiss to keep the science light; Harper’s Bazaar readers were not science fiction people. The constraint may explain why the story leads with loneliness rather than circuitry, and why it has aged better than most of its contemporaries. The technology in it is set dressing. The ache is the architecture.

And Kubrick did not find “Supertoys” through Aldiss’s fiction at all. He had read Billion Year Spree, Aldiss’s critical history of science fiction, and liked how he came off in it. He telephoned. They lunched, twice, before Kubrick asked Aldiss to send him some books. From the three that arrived in the mail, Kubrick chose “Supertoys.” So the director came to the story by way of the author’s criticism, with the fiction following after.

The collaboration that followed ran on a domestic rhythm Aldiss recalled with some warmth. A car collected him each morning, a few hours of serious work at Kubrick’s house, then lunch that Kubrick cooked himself, usually steak, with nothing stronger than coffee on the table.

The development stretched across decades, outlived Kubrick, and the film finally arrived in 2001 under Spielberg’s hand. The story about a manufactured boy waiting to be loved spent thirty years waiting to be made.

Don’t forget to read part two of this article - AI Toys: The buzz and the boundaries