If AI Slop Falls in the Forest

AI-generated writing gives me the ick. Spend enough time using LLMs and you too can spot the signature patterns of word slop. It’s not just possible—it’s easy. But even if the usual verbal tics aren’t on display—even if it aces the imitation game—it’s still a no for me.

Is this just a subjective preference? I don’t have the same visceral reaction to AI-generated images. I even find some Midjourney and DALL-E creations beautiful. But is that only because I happen to lack reverence for visual art, while prose is what I do to make (eke out) a living?

These facts might explain my (alleged) aesthetic error about image slop (along with my disastrous wardrobe), but they’re not the whole story. Like Kenny Easwaran, I’m not much bothered by the aesthetic flaws of AI-generated text when I’m conversing (directly) with an LLM.

Transactional writing also gets a pass. If I’m reading a formal email for work, I’m not trying to commune with another soul. Unless the sender is a friend, I just want clarity and cooperation. It’s fine, advisable even, to use AI to draft or revise work emails—especially if you’re like me and could sometimes stand to be more diplomatic. (Knowing you’ll offload also reduces stress. Chill, Claude will take care of it.)

Yet for creative expression—from books to tweets—authenticity matters. And disclosure doesn’t cut it. When it comes to fiction, personal essays, literary criticism, political diatribes, or any writing that carries voice and style, the words have to mean something—not just what the dictionary says but what the (human) author intended.

As it stands, LLMs aren’t very creative. But I’d still be averse to AI-generated creative expression (or its simulation or whatever) even if the words themselves were indistinguishable from a human’s—even if they were better. Some adopt this stance for bad reasons—think of creatives who just want to protect their turf. But there’s also a good reason: to make contact with another conscious mind.

Yet what if we’re talking about not creative expression but knowledge dissemination? Scientific papers are the clearest example, but this category—call it “knowledge writing”—also includes Substack posts that give it to you straight and news reports shorn of personal opinion. In this genre, what matters is the message, not the messenger. If the ideas are weak or the prose is sloppy, forget about it. Otherwise, though, what’s wrong with using AI to generate and absorb knowledge?

That’s one reason some readers might shrug at Substack’s new Pangram integration, announced by CEO Chris Best. (Click the three dots in the top right and find out, with a high degree of certainty, how much of a post was AI-generated. Go ahead, check.) Assuming a Substack post is knowledge writing rather than creative expression, why does provenance matter? If the signature patterns are there, you don’t need Pangram to tell you the prose is bad. And if not, Pangram doesn’t tell you anything you need to know.

“I came across a sound argument, but unfortunately I couldn’t endorse the conclusion because it was AI-generated.” You should just care about the words, no? If you dismiss the ideas only because an LLM produced them, then you don’t really care about knowledge, you just have a parasocial relationship with authors.

Along similar lines, Alexander Kustov defends using AI to produce knowledge writing: “If I ask AI to describe a chart, write a methods paragraph, or translate a regression result into normal English, the important question is whether the output is correct.”

Even skeptics agree. Here’s Eli Stark-Elster, in an otherwise withering takedown of AI-generated creative expression: “I don’t care whether humans take charge of articles that mainly exist to transport information. … I want all societal functions executed as well as possible. If AI executes some of them better, great.”

Not all cases are so straightforward. Some AI-assisted essays aren’t suffused with word slop, but you’ll still run across the odd phrase or sentence that you can’t quite (uh) grok. These errors make the essays worse, but they’re also a signal that you shouldn’t trust the rest. Like, there’s probably some other nonsense you missed.

Eric Schwitzgebel raises another criticism: the fact that AI wrote something is “higher-order evidence” that the expressions weren’t carefully chosen. This is true even if the author/curator vets every line. Schwitzgebel’s examples are compelling, though you also wonder whether the sand machines are sometimes more careful than the meat machines.

Schwitzgebel goes on to argue that academic journals should ban AI-generated essays. But journals are supposed to screen off higher-order evidence like the author’s identity. And if the difference is so subtle that an editor can’t tell anything’s amiss, maybe it’s not important enough to justify a strict rule. Then again, if journals are being overwhelmed by slop, a ban may be necessary to defend peer review, as Seth Lazar suggests.

Perhaps these objections miss a more important point. My friend Jacob Barrett argues that given the current state of AI, a human touch is still a good proxy for what’s worth reading. The essays LLMs now produce can seem superficially cogent but are intellectually shallow. If you find out it’s AI, that’s a good sign not to waste your time. So maybe a journal ban is justified. And even for knowledge writing, Pangram integration on Substack is useful after all.

Anyway, this line of criticism applies only to current frontier models. Future models will be better—and if sufficiently reliable, they might warrant trust without a human taking accountability for each one. It may be a while before you can upload a hypothesis and some data, tell Claude not to make any mistakes, and get a decent scientific paper as output. But I suspect solid AI journalism is not far off, if it isn’t already here. The machines are coming for AP writers first.

What’s more, maybe handing knowledge writing to AI will be not just acceptable and more efficient but epistemically superior.

As Dan Williams argues, LLMs are already more reliable than many other sources of politically relevant information. Social media has democratized punditry, which now better reflects popular opinion, but it has also made political discourse dumber. LLMs, by contrast, are “much more accurate, evidence-based, and in line with expert consensus.” Whether there’s audience demand is a separate question, but Williams is right that LLMs would improve public knowledge. Though for the self-standing essays they produce to have the same advantages, you’d also have to be able to prod them with follow-ups.

Dylan Matthews makes a related argument. Some communication technologies—from the printing press to social media algorithms—exposed people to different perspectives and increased divergence of opinion. But like network TV news, LLMs are “epistemically converging: they help homogenize the perspectives the population experiences and build a less polarized, more shared reality.” AI might therefore reduce polarization, marginalize conspiracy theories, and foster the common ground that is necessary for a functioning democracy.

Williams himself worries that LLMs might be too homogenizing. We don’t want people to inhabit contradictory worldviews, but we also want to preserve intellectual diversity. Partly to avoid settling too early on views supported by misleading evidence, partly because disagreement is constructive. I’m not sure LLMs would produce smoothbrains—not if they present both sides of contested issues. Or, as Williams speculates, leading LLMs (and their essays) could develop “different ideological valences.”

I’m nagged by a more familiar problem, one that afflicts writers rather than readers. What does all of this do to human agency? Writing is thinking, of course, and if we offload more writing to AI, will we do less thinking for ourselves? But that’s a topic for another essay—unless the machines get to it first.

Thanks to Jacob Barrett for helpful feedback. This essay was made possible through the support of Grant 63658 from the John Templeton Foundation. The opinions expressed here are those of the author alone.