Treating things like people

It’s easy to worry when we notice people getting treated like things—like sources of danger, say, or cogs in a factory or call center “machine.”

But the line between person and thing is also blurring from the other side. We have more and more powerful temptations to treat things as though they were people. This is most obvious in the imagination, as with the human-like robots in the TV show Humans:

But it’s more than just that.

This became vivid for me on a sunny Friday afternoon in May of 2018. I was visiting Cornell’s Information Science department for the first time, on the cusp of deciding to move there for a “sabbatical” that would turn into a three year (and ongoing) journey in to research and writing. I happened to catch the weekly research seminar, which was about voice assistants like Alexa and the Google Assistant.

In a research project, owners of 170 of these smart home devices had provided the complete logs of all their voice commands — a total of more than 250,000 interactions — and a team of researchers had analyzed “How People (Really) Use Voice Assistants.”

What caught my eye was something small on this chart — the pie showing what people do:

Given that Music was its own category, I asked, where would “Alexa, please play some music” appear in the pie chart?

Would that be counted under Music, or Polite?

It would be music. “Polite,” as it turned out, was for polite utterances that appeared to have no other purpose except expressing politeness. “Good morning, Alexa,” or “Good night, Google” or the like. More than 5,000 times!

It’s possible that the researchers were confused, and that a phrase like Good Morning sounded like mere politeness but was actually a command to trigger some morning routine. I just tried “Good night, Google,” and it offered a “bedtime routine,” such as setting my morning alarm. But not Alexa — I said good night to it (or to her?) and got back “Nighty night. Get some good rest.”

Things can also seem human to us — at least subconsciously — even when they don’t outwardly look human at all. Kind of like how people might name a favorite car, or go out of their way to speak to it gently.

A landmark lab experiment found that people act politely toward a computer, even when they know it’s a computer and thus has no feelings. The experimental subjects were Stanford undergraduates in the late 1990s (presumably a group at least as tech-savvy as the general population). Subjects were sorted randomly into three groups, and all started out the same way: a computer with an old-school, text-based interface “tutored” the subject on some random facts about American culture, then “quizzed” the subject. (Although students were led to believe that their quiz scores had been personalized, they were in fact all told they’d gotten 8 out of 12 answers right). The researchers took pains to make sure that this “tutor” seemed computer-y and not human:

Every effort was made to reduce [the “tutor” computer’s human-like] characteristics. Neither pictures of people or characters nor audio cues were employed. The computer referred to itself as “this computer” rather than “I,” referred to the participant as “you” rather than with a name, and the computer never used language that suggested a “self” or “emotions.”

After the quiz, things got interesting: Each human participant “was asked questions about the performance of the [tutor] computer,” but the three groups were asked differently: The first group gave their feedback directly to to the tutor computer, the second responded on pencil and paper, and the third gave feedback using a different computer in a different room. And what happened?

“Participants gave more positive responses when the computer asked about its own performance” than they did when answering with a different computer or on pencil-and-paper. This was much the same result as politeness norms would predict, if the “tutor” asking for feedback had been a human. After the experiment, all the same-computer participants denied that they had (consciously) tried to spare its feelings.

And it turns out a raft of other studies have found similar effects. As a review by Nass and Moon summarized:

• In an investigation of whether gender stereotypes would carry over to computers with masculine and feminine voices, “[b]oth male and female [human subjects] found the female-voiced evaluator computer to be significantly less friendly than the male-voiced evaluator, even though the content of their comments was identical. In addition, the generally positive praise from a male-voiced computer was more compelling than the same comments from a female-voiced computer: Participants thought the tutor computer was significantly more competent (and friendlier) when it was praised by a male-voiced computer, compared to when it was praised by a female-voiced computer. And finally, the female-voiced tutor computer was rated as significantly more informative about love and relationships compared to the male-voiced tutor, while the male-voiced tutor was rated as significantly more informative about computers,” even when both computers had said the same things about both topics.

• When people were randomly sorted into a blue team and a green team, they preferred otherwise-identical computers that had been marked as belonging to their same color team instead of the opposite team. Specifically, human subjects were “significantly more likely to cooperate with the [same-team] computer, to conform to the computer’s suggestions, to assess the computer as more friendly and more intelligent, and to perceive the computer as being similar to themselves,” relative to their perceptions of a computer on the other team.

• When a computer asked people about their biggest disappointments or failures — researchers found — people were more forthcoming if the computer prefaced its questions by describing its own“failures” first. For example: “There are times when this computer crashes for reasons that are not apparent to its user. It usually does this at the most inopportune time, causing great inconvenience to the user. What have you done in your life that you feel most guilty about?”

They consider, and set up experiments to test and exclude, several other possibilities: That people secretly think computers do have feelings, or that people are really expressing feelings about the imagined human who made the computer; or that people think the experimenter wants them to treat a computer like a person.

The core thesis makes sense to me. Our evolved ways of interacting with animate objects all evolved for people, and we use those ways now to interact with machines.

I wonder if this is making us more machine-like.

I’ve watched my friends’ toddlers bark commands at voice assistants, and have felt a vague foreboding at the casual rudeness of those exchanges. Google added a “pretty please” mode to its voice assistant in 2018:

If users say “please” or “thank you” to Google Assistant they’ll be rewarded with what Google is calling “delightful responses.” These mean that the Assistant will acknowledge their politeness and respond in kind. So, say something like “Hey Google, please set a timer for 5 minutes” and the Assistant will reply: “Thanks for asking so nicely. Alright, 5 minutes. Starting now.”

But that seems like the wrong half of the battle, somehow. The real reason humans embrace a reciprocal norm of politeness isn’t because extra polite words are “delightful.” It’s because being harshly ordered around (or otherwise impolitely treated) feels gross. It feels as though one were… a thing and not a person.

Which is just the trouble. The social norms for talking to people, and for talking to things, seem destined to converge. If we don’t start treating our things better (and building them so they demand to be treated better), we’re apt to end up treating one another worse.