Don’t Get Data Poisoned

Be careful, data poisoning is a thing.

Many people are incorporating generative AI (like ChatGPT) into their work. We have heard in the news about Artificial Intelligence making up fake legal cases or simply answering the simplest of questions incorrectly.

There is now a further threat: intentional poisoning of the data AI uses.

According to Jackie Snow, a tech writer at the Wall Street Journal, data poisoning is, “when malicious actors insert incorrect or misleading information into the data used to train an AI model with the aim of spreading misinformation, undermining the chatbot’s functionality or getting it to do something bad, such as share sensitive information.”

Hackers could place erroneous information on websites to sway the answers AI generates about Christianity, a political figure, or public policy. AI may grab the wrong information about how to fix your brakes, and thus putting your life at risk. They could train chatbots to lure someone to share personal data later used for nefarious purposes. The opportunities are many.

Actually, data poisoning is nothing new

The same accusation of data poisoning has long been made with the formation and interpretation of the Bible.

Muslims view the Bible as a corrupted document due to the way in which the texts came to be, and the interpretation of texts through the centuries. The poisoning of Jewish and Christian scripture, therefore they say, makes the Quran all the more inspired and authoritative.

The doctrine of Biblical inerrancy developed in the late 1800s as a reaction to perceived data poisoning churchmen saw from progressive European scholars. More than religion, every academic field or pursuit of truth has someone arguing for new or different data sources to get to the truth. Our national conversation about much of anything in the media, especially political, is seen by the other side as data poisoning to push one’s agenda.

Data poisoning isn’t anything new.

The scary part with generative AI is this: the poisoning is all the more secretive and less visible in the vast array of the internet.

What to do?

Wikipedia’s answer has been to democratize the collection and preservation of truth. Anyone can add to a Wikipedia article. I have. Those additions, however, can be changed by another person, and on and on. Eventually a team of volunteers will determine what is published if a topic becomes controversial. Their innovative democratization of truth has held up pretty well.

The best approach through human history, however, has been to create institutions.

Institutions determine, protect, and propagate truth. Religious institutions, libraries, museums, universities, publishing houses, think tanks, research labs and more have been human civilization’s way to generate and protect truth. And if data poisoning occurs (also known as reform, revolution, enlightenment, or just plain corruption) institutions either shift, die, get replaced, or continue in error. Most often, though, the good ones survive because the truth is plain to see by outsiders.

Fuller Theological Seminary and Westminster Seminary, an array of Christian denominations, and many colleges and universities emerged when one group saw the corruption of truth and the need to preserve what they knew to be truth. Political parties do the same in their own way, as do the institutions that grow up around the arts (The Met, Lincoln Center, The MOMA, and the Joffrey Ballet)

Our postmodern cynicism is quick to point out that all institutions are fallible and will sacrifice the truth for power or preservation. This is true. Yet, they work over the long haul, and are the way to carry human wisdom from one generation to the next.

So, do we need an institution to battle the data poisoning of generative AI?

I say yes.