Substack AI Detection Is Here. I Built a Skill For It.

Substack shipped AI detection on July 21.

You can scan any post, note, reply, or comment now. You get an estimate of how much a person wrote, and how much came from a model.

I want to be clear about where I stand before anything else.

I think this is one of the best things a platform has done in the last two years.

A lot of people spent this week hedging. Writing careful posts that agreed with the idea and then spent nine paragraphs undermining it.

I’m not going to do that.

Chris Best made the right call, and he made it early. That’s the part that deserves credit.

Every other platform waited until the damage was done and then announced a policy nobody could enforce. He moved while Substack was still clean, while there was still something left to protect.

That takes more conviction than acting after the fact. Acting late is damage control. Acting early is a bet, and you can be wrong in public.

Chris Best@chrisbest

We built the ability to do a @pangram scan into the @Substack app, because we’re sick of slop and we don’t want substack to turn into LinkedIn

4:16 PM · Jul 21, 2026 · 198K Views

71 Replies · 78 Reposts · 1.11K Likes

The internet is filling with text nobody wrote, for nobody to read.

If that becomes the default, the whole thing stops being worth showing up to.

So, I’m with him on this. Fully.

But I’ve spent the week reading the reaction, and there’s a problem underneath it that almost nobody is solving.

Writers I read are getting flagged. People whose work I know is theirs.

They’re posting about it. Some are frustrated. A few have started writing worse on purpose to score better.

That’s the failure mode.

Not AI slop. People sanding down their own voice to satisfy a classifier.

Here’s what the score can’t do. It measures how the words were produced. It cannot measure whether anyone thought anything.

That gap is the whole edition, and it’s what I built this week’s skill to close.

• What the detector actually scans: exact scope, the limits, the two things it can’t see

• The two kinds of AI content: one should be gone tomorrow, the other is how most good work now gets made

• How I build every edition you read: the real process, including where AI sits in it

• Your Take: the Claude skill I built this week. Not a bypasser. It hands your draft back with numbered markers where your own experience is missing, and adds your words exactly as you type them

• A real run: I tested it on an article, and I’ll show you exactly what came back

• The “How I make this” note: the Substack feature most writers missed this week

• Where this is going next: the pattern behind the launch that almost nobody covered

The download link is at the very bottom.

Substack is a writing platform.

That’s the uncomfortable part.

The thing AI does best is produce competent prose. So the one skill this platform exists to sell is the one most exposed.

If a subscription buys you nothing but generated text, following anyone is pointless. You’d query the model directly and skip the middleman.

Best’s framing gets this right. His argument isn’t that AI use is bad it’s that there’s a mismatch when a reader invests attention in something with no human thought behind it.

He called it Claudefishing.

That comparison is the sharpest thing anyone wrote all week.

The Mechanical Turk was a fake chess machine with a person hidden inside it. Claudefishing is the same con, running the other direction.

Now here’s the number that surprised me.

Pangram studied over a million posts across platforms. Substack was the cleanest long-form platform they measured, just under 22% flagged as AI-generated or AI-assisted.

LinkedIn was over 40% fully machine-written.

So Substack isn’t fixing a crisis.

It’s protecting a lead.

That’s a much smarter time to act, and it’s exactly why I think Best got this right.

Let’s get into what it actually does.

Scope first. Half the confusion this week came from people testing it wrong.

It works on:

• Posts, notes, replies, comments

• Text over 100 words

• Content published after July 21, 2026

• Web and iOS (Android coming)

It does not work on:

• Anything under 100 words you’ll get “not enough text”

• Video or audio posts

• The email version of your post

• Standalone Substack sites outside the app and Reader

Now the part that matters.

Two things it cannot see.

It reads the text, not the origin.

You can dictate your own hard-won method into Claude, have it arranged into clean prose, and still score as assisted. Because it was assisted.

The score is accurate. The conclusion a reader draws from it is wrong.

Effort is invisible to a classifier.

Fifty hours of work and a two-minute prompt can land on the same number.

⚠️ Detectors also have a fairness problem worth knowing about. A 2023 Stanford study ran seven detectors against essays written by people who speak English as a second language. On average, 61% of those genuine human essays were flagged as AI. One detector flagged 97% of them. Pangram is newer and stronger than anything in that study, so don’t read it as a verdict on this tool. But if English isn’t your first language and you write formally, you’re more exposed than a native speaker writing casually.

💡 Run your own published work through it before you form an opinion. Not to game anything, just to know your baseline.

Almost every argument this week collapsed for one reason.

People are describing two completely different things with one word.

Paste a link. Ask for a post about it. Publish whatever comes back.

No idea. No experience. No judgment.

Nobody read it before it went out. Not even the person whose name is on it.

This should die, and the detector will help kill it.

I have no sympathy here.

If you won’t read your own post, don’t expect anyone else to.

You have the idea. The method. The thing you built at 2am that nobody else is doing.

You dictate it, messily, in your own words.

AI turns that mess into something skimmable.

Then you rewrite it. Then again. Until it’s right.

That isn’t cheating. That’s what the tool is for.

The first type has nothing behind it.

The second has everything behind it and needs help with the last mile.

A score can’t tell them apart.

Which is why the burden is on us to make the difference visible.

I’ll show my hand, since I’m asking you to be transparent.

I have over 100 Claude skills, 50+ NotebookLM notebooks and many applications

I built every one myself, across hours nobody paid me for. Most of what I publish is me handing one of them over.

That’s the part AI can’t produce.

Not because a model couldn’t write about a skill. It could.

But the skill has to be built, broken, rebuilt, and tested against a real problem before there’s anything worth writing about.

Do I use AI? Yes.

Here’s exactly where.

Where AI is in my process:

• Turning my dictated, unstructured explanation into readable sections

• Tightening prose that runs long

• Catching things I skipped that a reader would need

Where it isn’t:

• The idea

• The build

• The testing

• The opinion

• The final read

I talk. It arranges. I rewrite until it sounds like me again.

💡 My tell that a draft got away from me: I read it back and can’t remember writing a specific paragraph. That paragraph always comes out. Every time. Most reliable quality check I have, and it costs nothing.

Which is where the skill comes from.

I need to be blunt here, because there are a dozen posts this week promising the exact opposite of this.

I’m against that, and I’ll say why plainly: text engineered to pass a scan while saying nothing is worse than slop. Slop is at least honest about being empty. Engineered text looks like effort.

Also, it doesn’t work. Pangram held up against three hours of deliberate attack from someone who does this professionally.

That’s the honest word for it. Not a trick. Not a workaround.

Convenience.

Here’s the problem it solves.

You finish a draft. You know somewhere in it there are places you should have said more of your own experience, your own objection, the number from the time you actually did this.

But you just wrote the thing. You can’t see them anymore.

So you either publish it hollow, or you spend an hour re-reading your own work hunting for holes.

Your Take does the hunting.

It hands your entire draft back to you. Every word, nothing rewritten. And wherever your own thinking is missing, it drops in a numbered marker with one specific question.

You reply by number. Your answer goes in exactly as you typed it.

Not paraphrased. Not tidied. Not corrected.

If you make a typo, the typo stays.

That last part is deliberate. The second it fixes your sentence, the sentence stops being yours, and the entire point was to get you into your own piece.

I tested it on a draft called “The 5 Best AI Tools to Use Right Now.”

Generic topic on purpose. That’s the kind of article most likely to come out hollow, and the hardest test for a skill like this.

Here’s what came back.

Notice what it did.

It didn’t summarise my article. It handed the whole thing back, every word in place.

It didn’t give me a list of problems. The marker sits inline, exactly where the missing thing belongs.

And look at the question it asked.

The draft had a section called How These Were Picked, with three criteria still used after month three, does something the others don’t, available today.

Reasonable-sounding rules. Written by nobody.

The marker said:

[1] YOUR INPUT
Missing: no origin this rule sounds earned, but nothing shows it was.
Q: Which tool did you use hard in week one and stop opening by month three?

That’s the whole skill in one question.

I had written a criterion that implied experience without containing any. It caught it and asked for the specific tool that taught me the rule.

💡 Answer the markers fast and messy. Don’t compose. The whole design assumes you’re typing the way you’d tell a friend, and the skill is built to keep that rather than clean it up.

This isn’t cheating.

Nothing about it produces writing for you. It cannot fill a single marker on your behalf

I built it to refuse, and if you skip a marker it deletes that passage rather than inventing something.

All it does is find the places where you should be, and get out of the way.

Substack shipped more than a scanner. It got buried.

When you publish, you can attach a “How I make this” note. Your own statement of process, in your words, visible to anyone who scans you.

You can also report and remove a scan of your own work you believe is wrong.

That second one matters if you get falsely flagged.

The first one matters for everyone.

I’d rather tell you exactly how I work than have a percentage tell you something slightly wrong about it.

So mine says what section three says. I build the assets. I dictate the explanation. AI arranges it. I rewrite until it’s mine.

Write yours today.

It takes four minutes and it ends the argument before it starts.

And here’s the part most people covering this week missed.

This isn’t a one-off reaction to AI.

Six weeks before the detector, Substack shipped Reply Rules: you set the rules for your own comment section, and they get enforced automatically.

Chris Best@chrisbest

On Substack you can now set your own Reply Rules that get automatically enforced. Your house, your rules. We don't have to choose between freedom and civilization.

Substack @Substack

The comment section of your Substack is all yours, to govern as you choose. Today we’re making that easier, with Reply Rules. When you set your Reply Rules, they appear to anybody replying to you, anywhere on Substack. And you don’t have to shoulder the moderation burden alone.

8:23 PM · Jun 3, 2026 · 24.9K Views

27 Replies · 3 Reposts · 12 Likes

Same philosophy, different feature.

Not a platform-wide ban. Not a rule handed down to everyone.

A setting you control, on your own publication.

Reporting on the launch suggests Reply Rules preferences for AI content are already being considered, which would merge the two directly.

So the direction is clear enough to plan around.

Substack isn’t going to decide whether AI writing is allowed.

It’s going to keep handing you the switches and let you decide what happens in your house.

Which means the writers who set their position early publicly, in their own words, will be the ones who never have to defend it later.

The score is not the goal.

Optimise for it and you’ll write shorter sentences, delete your em dashes, and produce something that passes while saying nothing.

That’s worse than slop. Because it looks like effort.

Write the thing only you could write.

Let the number land where it lands. Attach your process note.

That’s the whole strategy, and it survives every future version of every detector.

Install:

1. Download the file below