TLDR: Substack now lets readers scan posts with a tool called Pangram to estimate how much of the text was written by AI. The principle is sound, the philosophy behind it nuanced, but the execution is incomplete. An AI score cannot tell you whether judgement, conviction, and mission are in the work. Use it as you will, but know the limits. And learn the 3 ways to keep yourself in the words, whether you use AI or not.
For two years, I wrote slop for a living.
Dozens of articles a month on best laptops for students, best paints for your interior, best paints for your exterior. How to unclog a drain. Ten foods that help you sleep. All that and more.
I got a brief with a topic and a list of keywords, and I packed those keywords into the text until Google was happy. No AI anywhere near it. Just me, a list of words, and a deadline.
When AI got good enough to do that job, it did. My kind of work was one of the first things to go. And rightly so.
The keyword-stuffed article I used to create is table stakes now, automated away. It was pattern with nothing underneath it. A machine could do it because there was nothing in it that needed me.
I wasn’t in those articles. My judgment, my voice, wasn’t.
I think about those two years a lot this week, because Substack just did something big.
New here? I’m James, and I help founders and creators build AI systems that make them impossible to replace. My mission is to help you use AI to become irreplaceable at what you do by turning your expertise into systems no one else can copy.
My forever free Substack publication is where I, and the experts I invite, explore that idea of using AI to become irreplaceable.
If you want the foundation this piece sits on, start with How to Train AI on Your Brand Voice.
On July 21, Substack’s CEO published a post called “Against Claudefishing” and rolled out a detection tool built with a company named Pangram. Readers can now scan a post, a note, or a comment and get an estimate of how much of it a human wrote by hand.
The feed lost its mind. People read this new Substack AI detector as going to war on AI, the start of a witch hunt, a machine that would drag honest writers into the town square for using the same tools everyone uses.
It was, of course, more nuanced than that.
For one, Best was careful. He said the problem is not AI. He even explicitly said not every AI-assisted piece is slop.
He said Substack is not against people using AI to help their work, and that writers should get to pick their own tools.
What he named as the crime was very specific. He called it Claudefishing, the act of faking a human on the other end of the words, so a reader spends real attention on something with no real thought behind it.
To combat this, the Substack team introduced two new features: integrating with Pangram AI detector, and a disclosure statement. Theoretically, these will give readers the transparency that they need to combat “Claudefishing.”
And for the most part, I agree with this. I have watched that con fill timelines for two years. “Claudefishing” is the same emptiness I used to manufacture by hand as a freelance content writer, only now it scales with AI. It’s what my whole brand is categorically against: promptability, replicability. Also known (in my head) as the Gray Sea Of Mediocrity, also known more simply as AI slop.
But while the principle is sound, I’m not sure that the steps are enough.
I agree with the principle, but I also understand those who don’t. I believe everyone’s unease is about what people will do with the tool more than the tool itself.
A detector in the hands of a crowd that already suspects everyone becomes a weapon for accusation. It’s always the way this goes with AI detectors. This has happened to me: countless employers and partners have implemented “Must be below X% in [insert AI detection tool],” ruining the (completely human, btw) quality of the original draft. It’s happened in school, as well, with students penalized from AI detector false positives.
Best was careful about this, but the narrative will get flattened. “Scanned as AI, therefore fake.”
That is the default this drifts toward unless the narrative about AI, and AI writing, shifts. If not, this would set the conversation back a long way.
The scan can see the tool being used. It cannot see you.
Pangram checks one dimension of your writing.
It reads your words and estimates whether a machine shaped them. Plenty of tools do that. Pangram is one of the better ones, at least by its own 2024 technical report, which claims near-perfect detection across a wide range of writing.
Moreover, Pangram has been independently assessed against other commercial detectors. Researchers Brian Jabarian and Alex Imas via the National Bureau of Economic Research (NBER) published a September 2025 study specifically benchmarking Pangram vs Originality AI, GPTZero, and an open-source RoBERTa baseline. It was the only detector that kept policy-grade error rates across all four AI models tested.
Though, it’s still not perfect, as Yana G.Y.’s own testing, and Ruben Hassid’s experiment shows (btw, he’s offering a free skill in that article).
It’s just … not perfect.
So yes, Pangram might be something … but so what?
These tools can tell you if AI is used. But it cannot tell whether your judgment is in the sentence. It cannot tell whether the piece is aligned with what you stand for. It cannot tell whether a reader on the other end is spending their attention on a real point of view or on filler dressed up to look like one.
I am proof of this gap, from before AI was even here.
I wrote soulless, hollow work for years with my own two hands, and no detector on earth would have flagged it, because a human typed every word. The absence of a machine did not make it real. It was slop with a pulse.
The gap runs the other way too.
A writer can sit with a model open, argue with it, throw out most of what it says, and end up with a piece so full of their own conviction that it could not have come from anyone else. The tool was used, yes, but only to amplify the humanity.
A score tells you a tool was involved. But that should never be the only benchmark.
Yes. Authentic AI writing happens when the work stays aligned with who you have said you are and what you are for. A detector reads the tool. Your judgment and your mission are what make the writing yours.
If the tool isn’t the measure, something has to be. The standard I use held up long before tools like Pangram existed.
Your writing is authentic when it stays aligned with who you have said you are and what you are for.
Your promise, in other words. The reader came to you for a specific way of seeing the world. Authentic work keeps that promise. Hollow work breaks it, whether a human or a machine did the typing.
Most of what any of us make is repeatable pattern, the parts that follow a shape we have used a hundred times. That work can be handed off. The exception, the judgment call, the moment where you decide what is actually worth saying, stays with you.
Encode the pattern. Protect the exception.
A machine can carry the repeatable weight so you have more of yourself left for the part only you can do.
The standard is easy to say and harder to live.
Three habits keep me honest about it.
Critical thinking here comes down to knowing the trade you are making, and defending it out loud. When you can’t say why the model touched a given part of your work, that part is running on autopilot, and autopilot is where slop comes from.
Ask yourself:
Can I explain exactly where in my process I use AI?
Do I understand what I give up when I use it there?
Do I have concrete reasons for making that trade?
Alignment needs something to align to.
Until you have named your promise, you cannot tell whether a draft keeps it or breaks it.
Get specific about who you serve and what you stand for, then hold every AI-assisted choice against it.
Ask yourself:
Can I say in one line what I stand for and who I serve?
Does this piece keep that promise or wander off it?
Would the reader who came for me recognize me in this?
Your voice is how you show up on the page, separate from your goals, your motivation, or your mood on a given day.
A model will pull you toward its own smooth default the moment you stop paying attention. So, know the rules of your own voice well enough to catch them slipping.
Ask yourself:
Can I name the rules that make my writing sound like me?
Does this read like me out loud, or like anyone?
Did I let the model overwrite a choice I would have made?
Substack also gave writers an optional “How I make this” statement, a place to explain your process. Most people are treating it as a confession booth. That is backwards.
That box is the most unpromptable thing on your page.
It is where you get to say what you are for and how your hand shows up in the work. A model cannot fill that box in for you, because it has no mission and no scars to draw on.
Fill it in yourself.
Tell people what you refuse to automate and why. You will find it does more to earn trust than any clean detector score ever could.
People assume that using AI to write means teaching a model to copy your voice well enough to slip past a scanner. That is the shallow version, and it is the one Best is right to be against. Voice worn as a costume.
But there’s a better, more mindful way to do things. This is what I’ve tried to do.
I have encoded myself. My values, my worldview, my years of writing experience, the things I have lived through, my specific pet peeves, the principles I refuse to break. Those are the criteria my drafts get measured against before anything ships.
This is a long way past a bot that knows my posting schedule.
So when I bring AI into a piece:
It works like a mirror held up to my own promise
It checks whether the writing still matches what I stand for while I juggle a dozen things at once.
It helps me straighten a promise that has drifted.
It pressure-tests an argument from a reader who disagrees with me.
It widens my research past what I could gather alone, and it pushes a half-formed idea until it becomes a real one.
I’ve discussed AI writing in my Substack many times. I wrote my process for it here. I’ve also invited guests who showed their own processes:
Used this way, AI leaves more of you in the work. It clears away the mechanical parts, so your attention goes to the parts only you can carry. What reaches the page comes out more concentrated.
If this week rattled you, be plain with yourself about what the scan actually is.
It is not the judge of your worth as a writer. Run it on your own drafts if the number steadies you, then set it down and ask what matters:
Is my judgment in this?
Is it aligned with who I say I am?
Would a reader feel a person here?
When the answers are yes, publish and don’t flinch. Write authentically with AI, no matter what the numbers turn out to be.
The scanners will get better. The anti-scanners might get better too. The feeds will keep filling with hollow text that scales faster than any of us can write by hand.
None of that decides whether your work is real.
The way through is to be so present in what you make that the question of which tool touched it stops mattering, because a reader can feel you standing there.
I spent two years absent from my own writing, back when it was just a way to earn a living. I am never going back, and no button on a platform is going to send me there. If you are in your work, you are safe. Stay in it.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.