I had a deep, visceral response when I read about Substack’s new partnership with Pangram, an AI detection tool.
Oh, hell no, I thought. Which was weird because I’ve never been the sort of person who previously had strong feelings about either a) Substack’s growing list of features or its decisions as a company or b) even about AI general. (Like many of you, my thoughts are all over the place about AI. Up until about a year ago, I figured I — an annoyed Gen Xer — might just age out of ever needing to have an opinion or needing/wanting to use AI but alas things have evolved more quickly than I thought.)
Since then, like many of you, I’ve been trying to figure out why I’m so enraged by Substack’s decision.
It’s not just because it’s yet another example of our tech overlords messing up something good for profit.
It’s not only because I quickly learned and experienced for myself that the AI tool in question is often inaccurate — or that it disproportionately labels the writing of non-native English speakers, neurodiverse writers, writers of color, members of the disabled community, and women as written or assisted by AI.
It wasn’t just that I feared that this partnership would lead to witch hunts on this platform.
The primary reason for my anger is because my job here on Substack is to listen to all of you in building a community of midlife women. And I know how scary, how vulnerable, and how difficult it is for many of us to declare that we are indeed building something important as writers here — that our words and experiences as midlife women are valuable, that our ideas should be listened to, that we even want to be paid for what we can offer.
What I see from writers — particularly from our community — in my own chat thread on this topic (with over 175 replies and counting), in comments on my notes — is that many of us are angry because this move from Substack feels like another voice telling us not to trust ourselves and our own judgment, our own ability to discern what we want to read. For how many writers will this labeling (or fear of labeling) cause them to question their own voice — how they organize a paragraph, how they phrase a sentence, how they research their ideas? And how many will — like some here at Midstack have already told me — just leave Substack altogether?
A few comments from our Midstack thread:
“I just used three different detectors out of curiosity. I’ve never done that before. They all came back with different results. One was zero AI, one was 34%, and one was 59%. I use AI to help me organize my writing, but not for the actual content. That is mine. It seems like this addition will be doing a disservice to everyone!”
“I’m a writing prof and I can say that these checkers aren’t super useful. I’ve had plenty of student writing get flagged as AI written and I can tell that it wasn’t. Even when speaking with students who I suspect may have used it, it’s clear in talking with them that they didn’t. I ran part of something I wrote for a journal article through an AI checker and it came back as 65% AI—when clearly it was all human, all me.”
“I ran my own test. Content I wrote entirely myself, content I wrote with light AI edits, and fully AI-generated content all came back the same: 100% AI, every time. Other writers report the reverse, AI content scoring as fully human. A detector that flags everything, or nothing, is not detecting. It is guessing.
The risk is that real writers get penalized for a tool that cannot reliably tell the difference. That does not clean up the platform. It just adds noise and suspicion.”
After an old personal essay that I pasted into Pangram was judged to be AI-generated, I wrote a note about it and began to hear these concerns of other Substackers. Then Natasha Moni messaged me about working together to contact writers with experiences or expertise related to AI.
You can read more about our collaboration in reaching out to writers here: “Substack Writers Weigh in About AI Concerns, Biases” in Natasha Moni’s Substack Someday This Will Be Funny
I next reached out to Karen Smiley of Everyday Ethical AI and messaged her a few questions that she graciously agreed to answer for me.
Karen: One, that all AI detection tools, whether AI-based or not, have biases, and they are never going to be 100% accurate. (More about that below.)
Two, there is already too much AI shaming and judging. Substack has just given the ‘witch hunters’ and ‘purity police’ a new seemingly-quantitative weapon to attack people with. Having Pangram in the Substack UI is going to disrupt the writer-reader relationship and lull people into trusting the AI score, instead of using their wetware (human brains and judgment) to decide if an article or writer is worth their time to read. The scoring is likely to push people towards editing themselves to satisfy the scoring system and away from what works best for their ideas and their readers. That’s counterproductive.
Three, those of us who want to protect our writing and keep it out of the virtual hands of AI tools haven’t been given a real choice to consent or opt out of this use of our writing. Reassurances about our writing not being used by Pangram to train AI are still too fuzzy and not credible. For the first few days, there was literally no way to opt out of having one of our articles scanned by Pangram without first letting Pangram scan and score our article! We now have an article-level opt-out that doesn’t require us to let it scan. We should have it for comments too, and ideally, a newsletter-level or profile-level opt-out so we can ‘set it and forget it’.
Karen: First, think about what Pangram’s claims of 99.98% accuracy mean at the scale of Substack worldwide. That’s 1 in 10,000. How many millions of articles and longer comments get posted every week? At that rate, thousands of writers every week will be unfairly flagged by a false positive.
No AI detection tool, whether AI-based or not, is ever going to be 100% accurate. And they are known to be biased against certain patterns of writing. Examples include by race, ethnicity, gender, neurodivergence, academia, and second language writers. I and many others don’t accept Pangram’s claims that their tests adequately prove that it is highly accurate even on all of these subgroups. Writers I know who fall into these categories have tested their own human-written or even pre-genAI work on Pangram, and found it scored as AI-written. Minor tweaks to a text can cause dramatic changes in the scoring. Readers should not be trusting it more than their own human judgment.
Being falsely scored as AI-assisted or AI-written is going to be both more likely and more damaging to writers who are already, in many cases, disadvantaged. That’s not ok. And studies already show that women are penalized more heavily for using AI than men. That will just make it even worse.
Karen: No, I’m going to block Pangram scoring on every article I publish. I won’t be using Pangram on any of the articles or comments written by anyone I read, either. I published an article about this in my Everyday Ethical AI newsletter on Wednesday explaining why. In brief, I refuse to be a party to inflicting harm on other writers.
Karen: First, don’t despair. I know some folks have already faced harassment for AI use and the availability of Pangram could make it worse in the near term. I’m seeing enough of an uproar that I am hopeful that Substack will see the light and allow us to opt out more easily. Many authors like me who don’t use AI to write are blocking Pangram. Others who have environmental concerns about AI, and think it’s ridiculous to use AI to check for AI, are blocking it too. If enough of us opt out to *normalize* blocking Pangram, we can help to protect each other from unfair consequences of the Pangram scoring.
Please check out Karen’s recent post (and subscribe to her Substack):
“A tool that cannot produce consistent, stable feedback on trivial edits to human-authored text isn’t reliable for anything, let alone a result that could unravel an author’s reputation. A false positive like the ones I describe here could cost a writer a publication, a grade, a contract, or their credibility with an editor or reader base.
Meanwhile, the vendor of the tool faces no consequences for getting it wrong.
Yet institutions and businesses – like Substack – are already using tools like Pangram to make consequential decisions. This is genuinely dangerous, not just annoying.” — Read more:
“Then, there are the folks who are telling everyone to calm down because, as of July 23rd, you can just turn the option off2. Problem solved, right? Except we’re all witch hunters now, and everyone’s guilty until proven innocent. And turning the option off is surely not a thing somebody who is innocent would do now, is it? Hmmmmm?
And if you’re leaving the option to use the cheater checker ON, proud of your unassisted workflow and staring down the possibility of a false positive with the confidence of Chuck Norris, what happens when someone feeds your work to the machine? Substack says it’s not being used to train either company’s system, but Pangram’s privacy policy is nebulous, and I’ve been here for long enough to know that creators’ best interests are not Substack’s priority.” Read more:
“I Have Opinions, and They Are That This is Some Bullshit” by Jennie O'Connor of Down the Rabbit Hole
“If someone with dyslexia uses spellcheck, we don’t accuse them of cheating.
If someone who is blind uses a screen reader, we don’t accuse them of cheating.
If someone who has arthritis uses voice dictation, we don’t accuse them of cheating.
So why are we treating AI differently when it’s helping people with neurological disabilities communicate?
What that flag actually says, underneath the algorithm, is that a disabled person’s polished words can’t be trusted.” Read more:
“Substack Flagged My Story as 100% AI… and It Was Wrong” by Migraine Girl 🧠
“Pangram is a real machine, trained on vast amounts of text to make a probabilistic guess, and we have pointed it at your writing to work out whether a person is hidden inside. To decide if there is a human on the other end, Substack asks a machine.
We are outsourcing a judgement that was supposed to stay human. The reader used to decide whether a piece felt alive by reading it. That is the skill. Handing it to a detector retires the muscle we most need to keep.” Read more:
”Decisions get made about your work, your face, and your reputation, and you find out when it’s already live. Sometimes there’s a reversal, sometimes there isn’t, and either way you were never in the room.
Which is why I’ve gotten very clear that we don’t owe these platforms anything. Not loyalty, not an explanation, not a disclosure we didn’t choose to write. I am certainly not explaining myself to a stranger on the Internet either. Ever.
Substack is free right up until it isn’t. The content is the inventory, the writers are the supply, and there are investors somewhere who need this to work out eventually. I’m not mad the business model exists. I’m just no longer confused about who it’s for.” — Read more:
“Running a detector on every piece of writing on this platform is not free. It’s human’s energy and environmental cost spent on solving a problem that a human trust was already solving, for free, and without the exhaust.
We’re literally burning more power to mistrust what people were already evaluating with their own judgment.” Read more:
“I Don’t Work for the Scanner” by Anna • bubble boss 🫧 of How To Boss AI
💬 What are your thoughts and experiences? What do you think of this decision? Tell me in the comments.
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.