Why AI can't make us morally better

It’s been a long day of work. You get home, kick off your shoes, hurl your bag into the corner of your room, break out of your work clothes, and plop down at your desk – you’re stressed.

Ping! You’ve got a text from your boyfriend, and a text from your mother. You were supposed to be at your boyfriend’s apartment for dinner, but your stress caused it to slip your mind. The right thing to do is to scurry right there. However, your mother reminds you that you promised to help her move some boxes tonight.

A good ol’ moral conundrum.

Ethically reasoning in this situation is so hard, you’re stressed and tired. Don’t you wish someone could do it for you?

Well imagine you had an infinitely patient, infinitely compassionate, infinitely knowledgeable, and infinitely helpful AI reasoning bot which told you the ethically perfectly thing to do. It drafted apologies, weighed promises, and triaged obligations, all while you grabbed your keys to go. If this was possible, wouldn’t AI have the potential to make us better people?

This is what Marc Andreessen thinks.

In his paper, Why AI Will Save the World, his central claim can be found in the second line, “AI will not destroy the world, and it may in fact save it.” Evidently, Andreessen is a techno-optimist (he admits as much).

He predicts that AI will have sweepingly positive impacts on healthcare, education, daily life, work, – virtually any field you could think of. His confidence comes from AI’s rapidly improving ability to synthesize information and reason. Models like ChatGPT are reasoning at PhD-student levels across a variety of fields, and we have good reason to believe that they’re only going to get better.

It’s because of this fact that Andressen believes that every person will and should have an AI assistant to help navigate life… scientists will have agents that help them interpret data, consultants will have agents that help them think up unique and creative business solutions and so on. The common thread between these examples is productivity. AI will make us more productive. This presumption I do not doubt.

Andreessen doesn’t stop at productivity however, he postulates that AI will make us better people. After all, AI has the potential to be “infinitely patient, infinitely compassionate, infinitely knowledgeable, [and] infinitely helpful.” Prima facie this reasoning seems okay. If AI can reason and calculate at extremely high levels, then it should be able to tactfully weigh alternatives in morally difficult situations, do utilitarian calculi more accurately, and thus teach us exactly how to act morally.

Andreessen relies upon a substantial assumption – that concepts such as compassion, knowledge, and patience are measurable and deliverable – no matter the context. To further flesh out my objection, I will call upon the work of Logical Positivist Moritz Schlick, in his work, The General Theory of Knowledge. There is no single impression of “compassion.” So, we introduce a concept/sign, which has a definition that determines its intensionality. The meaning of “compassion” is found through listing the ways in which the concept is constituted, i.e. caring for one when they are down, buying one’s mother flowers on her birthday, etc. In this way concepts are analytic, or in other words, the intension of the concept is what the definition stipulates it is.

Historians of language have shown us that words evolve and change over time. Compassion today doesn’t mean what it did 100 years ago. Caring for another in a time of struggle is still considered compassionate, but we wouldn’t consider the act of withholding a cancer diagnosis compassionate, which it was considered 100 years ago. Philosophically, terms like compassion are thick concepts, meaning they have both descriptive and evaluative natures. Moreover, their context is fixed by rule-following and the patterns by which they are used.

Compassion doesn’t have some metaphysical essence that transcends communal barriers in the way that is necessary for Andreessen’s AI hopes to come to fruition. Practices and power relations in societies shift, and concept and language drift with it. Moral of the story, abstract concepts are socially governed rules for applying a sign to concrete behaviors under localized norms.

A consequence of this: correct word usage is cross-cultural and cross-communal. Therefore, the promise of an “infinitely compassionate” or “intelligent” AI is incoherent. At most a pupil of an AI tutor could specify bounded and culturally relevant behaviors which would give them a finite amount of “compassion” or whatever they seek, but not moral omniscience.

To conclude I want to make something clear, I’m not a Neo-Luddite anti-AI warrior. Actually, I think it has the potential to cure many of society’s ailments as Andreessen points out. It might help us cure cancer, produce more art, provide education to underserved communities, etc. But I don’t think it can do this one thing: make us morally better.

This piece was adapted from a writing assignment for Victor Kumar’s seminar on epistemic tribalism and AI ethics.