AI Is Changing Our Language (and How to Use AI to Measure That)

AI is changing how we write and speak.

So, for example, LLMs Distort Our Written Language: shows that “LLMs not only alter the voice and tone of human writing, but also consistently alter the intended meaning,” and “that extensive LLM use led to a nearly 70% increase in essays that remained neutral in answering the topic question. Significantly more heavy LLM users reported that the writing was less creative and not in their voice.”

This is probably obvious to many of you, but did you realize that if you’re spending time with LLMs, it might be distorting how you speak? They analyzed academic talks and podcasts and found “a measurable and abrupt increase in the use of words preferentially generated by ChatGPT, such as delve, comprehend, boast, swift, and meticulous, after its release.”

Armin Ronacher suspected the same, and worse:

The choice of words is one thing; the way in which LLMs form sentences is another. It’s not hard to spot LLM-generated text, but I’m increasingly worried that I’m starting to write like an LLM because I just read so much more LLM text. The first time I became aware of this was that I used the word “substrate” in a talk I gave earlier this year. I am not sure where I picked it up, but I really liked it for what I wanted to express and I did not want to use the word “foundation”. Since then, however, I am reading this word everywhere.

So Armin decided to find out which words are more likely to be used by LLMs. Before clicking on that link, think of how you would go about answering that question. If you’ve started using Claude Code or Codex CLI, Armin’s answer would seem pretty obvious to you:

What I did was take 90 days’ worth of my local coding sessions and look for medium-frequency words where their use is inflated compared to what wordfreq would assume their frequency should be. Then I looked for the more common of these words and did a Google Trends search (filtered to the US). Note that some words like “capability” are more likely going to show up in coding sessions just because of the nature of the problem, so the actual increase is much more pronounced than you would expect.

(it should go without saying that he probably used an AI coding agent to do all of this analysis.)

Click here to see/use the tool he built to find out which words are most used by LLMs in excess of their normal human language frequencies. (No, they are not necessarily the usual suspects like “delve”, but looking at the list is instructive.)

I wrote this article for two reasons:

• If you’re a heavy LLM user, you should be aware that LLMs can change the way you write and speak. So take steps to not fall into this trap

• I wanted to give a great example of how AI agents can make analysis easier. If you are curious about how something is trending, instead of just guessing, you can now use AI agents to get some decent evidence for it.

The second point is important: we all should start thinking with agents. It does not come naturally to us, but if we can do it, we have advantages in the coming turmoil.