Training on Noise

Apparently over half of new content on the internet now is AI slop. We are making more AI slop content than we are making our own real content.

I’m not super surprised by this, it’s just another step toward dead internet theory becoming reality. At this point, there is AI using AI to make more AI slop; just these bots communicating with each other. There are humans using AI to teach them how to use AI. There are both purposeful and inadvertent uses of AI (using ChatGPT on purpose versus Googling something and the first result being AI summary).

It’s just a steady crawl of AI taking over the internet little by little.

When you think about it, however, what does this mean for the AI itself? AI is, after all, trained on internet data. It constantly scans and scrapes the internet for content to analyze from which it picks up on patterns that it can repeat itself and create new content.

But, with over half of the internet already being AI slop content, AI will be using more of its own content to train itself than it will be using human content. AI is also indiscriminate. It won’t know to not pick its own content to train itself with. It’ll just keep recycling its own content over and over again.

So what does this mean for AI? I believe it signals that there will be a peak that AI will hit whereby it won’t improve anymore. Although it seems like AI is getting better by the day right now, the law of diminishing returns will hit it. And, if anything, like AI already does now, it will continuously exacerbate little nuances that signal that content is AI. Like in an AI video where someone’s mouth is just a little to funky looking and so is a giveaway that the video is AI. AI will just continue to amplify these features as it continues to train on its own content with these features already present. We might even see AI get worse should this continue.

Alternatively, AI developers will figure out how to account for this. They will figure out how to filter AI content out of the training data. Even then, I believe we will see a bit of a Zeno dichotomy paradox.

If the goal of AI is to emulate human behavior (writing, speaking, art, etc.) and the AI trains on our data but is still imperfect, it will need an infinite amount of data to train on to actually perfectly emulate us.

Say the AI is at 50% human emulation. The next step is 75% emulation, then 87.5%, and so on. Although the AI is taking leaps and bounds toward perfect human emulation, it will always be a little bit of a step behind.

I would say AI right now is probably at >99%, but it still just isn’t quite there. It isn’t perfectly emulating us yet.

Could we know, though? We ourselves aren’t even perfect. What would it mean for the AI to perfectly replicate the things we do? Or do them even better? Especially when it comes to creative tasks? If the AI did something “better” than us, we would not even know it. Although I am confident in this postulate that AI won’t be able to do things better than us qualitatively, if it did, we would not even be able to tell.

Regardless, why else am I not worried about AI so much? I take it kind of like drinking among young people nowadays. Drinking among young adults in the U.S. has subsided over the past couple decades. Why is this? Gallup has their own speculation that is measurable, but I think it is fair to say that young people are more health conscious than ever now. There are more people going to the gym, less people smoking, there is less juvenile offending in some capacities, less littering. These are all things that are subjectively bad and so people are doing less of them.

At the same time, research indicates that Americans are more concerned about AI than anything else. I sense that AI will soon join the group of things and activities listed above. People will start to think to themselves “hey, this AI stuff isn’t good for us” and so will gradually stop using AI.

Consider, as well, the future of the technology. Should it continue in the capacity that it is right now, it will be better and better at making visual media. I have speculated that you will be able to type one prompt into the AI, “make me a movie about X and Y,” and within minutes you’ll have a 2 hour long film about X and Y.

That sounds cool of course, but think about it: who will this benefit? The AI companies are going to make AI that is more efficient at making AI then they are. And the AI will be so efficient that it will no longer be profitable because the AI will require so much power to keep it running, draining the AI company, while simultaneously be used for content that they will not profit from. Movie studios are not going to profit from this. You will not profit from this. Premium AI accounts will probably cost an exorbitant fee that no regular person will and no serious creative would use as it, well, not creative.

This is all extremely cynical, and I trust AI will still have many great uses, but I think it’s that there’s an AI bubble right now waiting to pop. It’s not going to go away forever, but AI has a lot to sort out right now.