A society that wants to understand itself probably wouldn't act like this

In the 1960s, a German-American computer scientist named Joseph Weizenbaum coded an early version of today’s AI chatbots. Weizenbaum called his program ELIZA, after the “My Fair Lady” character Eliza Doolittle who takes speech lessons (and gets better).

How people reacted to Weizenbaum’s crude creation tells us almost everything we need to know about AI hype more than half a century later.

ELIZA could hold basic “conversations,” including playing the role of a psychotherapist with real human users. An example of ELIZA’s responses to one woman are shown in capital letters:

Anybody with a cursory awareness of recent headlines about AI romances and AI psychosis already knows where this is going. ELIZA’s human interlocuters in the 1960s, despite talking to a clunky machine they knew had been programmed by Weizenbaum, refused to believe that they were talking to a mere machine. His secretary, having watched him build the contraption over several months, after just a few exchanges with ELIZA, asked Weizenbaum to leave the room so she could have some privacy.

Weizenbaum, exhibiting a bit of Freudian sangfroid about all this, was not surprised to see people form emotional attachments with inanimate objects. He’d already seen people get attached to their cars or guitars or computers. But “what I had not realized is that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people,” Weizenbaum wrote in his 1976 book “Computer Power and Human Reason: From Judgment to Calculation.”

Two other responses to ELIZA disturbed Weizenbaum, a Jew who had fled Nazi Germany as a child. Some experts thought the machine could be perfected enough to supply actual therapy to real people, reducing the need for real therapists. Others thought the program demonstrated that computers could understand natural language, when Weizenbaum, who’d created the program, knew it wasn’t doing anything of the sort. It was becoming clear that the problem wasn’t the technology, but people’s flawed understanding of it:

This reaction to ELIZA showed me more vividly than anything I had seen hitherto the enormously exaggerated attributions an even well-educated audience is capable of making, even strives to make, to a technology it does not understand. Surely, I thought, decisions made by the general public about emergent technologies depend much more on what the public attributes to such technologies than on what they actually are or can and cannot do. If, as appeared to be the case, the public’s attributions are wildly misconceived, then public decisions are bound to be misguided and often wrong.

I learned about Weizenbaum’s ELIZA experiment from Neil Postman’s 1992 book “Technopoly: The Surrender of Culture to Technology,” a work of technoconservatism that, like Weizenbaum’s writings, was imbued with foresight about our struggles with today’s vastly more powerful technologies.

Consider this passage from Postman’s “Technopoly”:

In a technocracy, tools play a central role in the thought-world of the culture. Everything must give way, in some degree, to their development. The social and symbolic worlds become increasingly subject to the requirements of that development. Tools are not integrated into the culture; they attack the culture. They bid to become the culture. As a consequence, tradition, social mores, myth, politics, ritual, and religion have to fight for their lives.

Technology, attacking and taking over the culture? Bending society to its own imperative for advancement? In my United States of America? Postman (most famous for writing “Amusing Ourselves to Death”) thought the U.S. was the world’s first “Technopoly,” a society marked by “the submission of all forms of cultural life to the sovereignty of technique and technology,” where information itself has become a form of pollution.

To Postman, “the milieu in which Technopoly flourishes is one in which the tie between information and human purpose has been severed, i.e., information appears indiscriminately, directed at no one in particular, in enormous volume and at high speeds, and disconnected from theory, meaning or purpose.”

Neil Postman wrote “Technopoly” before the introduction of ChatGPT and Sora; TikTok and YouTube; Twitter and Facebook; Google Search and the Netscape browser. Postman wrote the book before Windows 95 existed. A philosophy of technology that mostly holds up through successive eras of technical revolution has already passed time’s first test, which is for the philosophy to outlive the philosopher. And Postman’s philosophy is ultimately conservative, motivated by the desire to preserve the traditions of humanism, social cohesion and a shareable sense of collective history.

Technoconservatism was old before it was new. Postman quotes Plato’s “Phaedrus,” where Thamus warns that whoever learns writing (one of our first dangerous technologies) “will cease to exercise their memory and become forgetful” and “will receive a quantity of information without proper instruction, and in consequence be thought very knowledgeable when they are for the most part quite ignorant.” And Postman, to his credit, is like — well, yeah! Writing really did that. A new technology is neither good nor bad, but ecological: it “does not add or subtract something. It changes everything. In the year 1500, fifty years after the printing press was invented, we did not have old Europe plus the printing press. We had a different Europe.”

In previous generations, societies dealt with information revolutions (which always produced information gluts) by creating institutions that prioritize “good” information and deprioritize the bad; think about schools with their organized curricula, courts with their standards of evidence, newspapers with their party lines or codes of journalistic ethics. But Postman notes that we got lucky after the Gutenberg revolution, when information technology’s development slowed down long enough for societies to catch up and be excellent:

From the early seventeenth century, when Western culture undertook to reorganize itself to accommodate the printing press, until the mid-nineteenth century [with the invention of the telegraph], no significant technologies were introduced that altered the form, volume, or speed of information. As a consequence, Western culture had more than two hundred years to accustom itself to the new information conditions created by the press. It developed new institutions, such as the school and representative government. It developed new conceptions of knowledge and intelligence, and a heightened respect for reason and privacy. It developed new forms of economic activity, such as mechanized production and corporate capitalism, and even gave articulate expression to the possibilities of a humane socialism. New forms of public discourse came into being through newspapers, pamphlets, broadsides, and books. It is no wonder that the eighteenth century gave us in the work of Goethe, Voltaire, Diderot, Kant, Hume, Adam Smith, Edmund Burke, Vico, Edward Gibbon, and, of course, Jefferson, Madison, Franklin, Adams, Hamilton, and Thomas Paine. I weight the list with America’s “Founding Fathers” because technocratic-typographic America was the first nation ever to be argued into existence in print.

Contrast the luxuriously slow social pace of the Gutenberg era with today’s information development timelines. Over the course of three decades, we’ve seen the rise and now-decline of the open web; the rise and now-decline of social media; the rise of short-form video and the rise of chatbots and synthetic information. All created enormous economic and philosophical disruptions whose fundamental impacts you can’t get a group of people in a room together to describe accurately. Among the disruptions: These increasingly efficient forms of sharing information keep encountering falling test scores; universities are trying to implement AI as their own students use it for cheating or boo the tech at their graduations; people are falling in love with their chatbots, which sometimes tell their users to kill themselves. A society that wants to understand itself probably wouldn’t act like this.

Neil Postman’s biggest criticisms weren’t of technology itself, exactly, though he was an adept of Marshall McLuhan’s influential formulation that “the medium is the message” — meaning that the form of communication dictates what is capable of being said. Postman was harsher on the instrumentalist view of technology that puts humans second. This instrumentalist view happens to be the organizing principle of a culture that has just made Elon Musk our first trillionaire while he used his social media network to help foment the race hate in Northern Ireland.1 When someone like Musk is both medium and message, there’s no point expecting a rotting pipe to do anything but spew its bilge all over everything.

“You must try to be a loving resistance fighter,” Postman wrote in the conclusion of “Technopoly,” describing the anti-technopolists as people:

who pay no attention to a poll unless they know what questions were asked, and why;

who refuse to accept efficiency as the pre-eminent goal of human relations;

who have freed themselves from the belief in the magical powers of numbers, do not regard calculation as an adequate substitute for judgment, or precision as a synonym for truth;

who refuse to allow psychology or any “social science” to pre-empt the language and thought of common sense;

who are, at least, suspicious of the idea of progress, and who do not confuse information with understanding;

who do not regard the aged as irrelevant;

who take seriously the meaning of family loyalty and honor, and who, when they “reach out and touch someone,” expect that person to be in the same room;

who take the great narratives of religion seriously and who do not believe that science is the only system of thought capable of producing truth;

who know the difference between the sacred and the profane, and who do not wink at tradition for modernity’s sake;

who admire technological ingenuity but do not think it represents the highest possible form of human achievement.

“In short,” Postman concludes, “a technological resistance fighter maintains an epistemological and psychic distance from any technology, so that it always appears somewhat strange, never inevitable, never natural.” This is technoconservatism: Standing athwart history, yelling for people to log off, or to at least stop falling in love with your computer.