AI Detection Has a Race Problem

As an educator and an author, I am following discussions about AI use in writing from two different angles. Recently I sat in a curriculum meeting as a teacher and engaged in a discussion about using AI plagiarism detection software in the school. One of the staff members had inserted a classic novel into the software to test its accuracy. It had reported that the novel was 90 per cent AI generated. If the classics can’t pass, the tool isn’t measuring AI, instead it is now measuring writing that is too well executed and too consistent.

Even though there have been many instances of AI software being wrong when labelling writing as AI generated, it is still being used to discredit writing that people label ‘AI slop.’ Recently, Jamir Nazir, a Trinidadian writer, won the Caribbean regional category of the 2026 Commonwealth Short Story Prize for The Serpent in the Grove, selected from 7,806 entries by a five-judge panel chaired by Louise Doughty, published in Granta. Within days there was a social media pile on that the writing was AI generated with the proof being “syntactical tics.” When Wharton Professor Ethan Mollick publicly called it “100% AI generated” it seemed the case was proved without a doubt. Or was it?

Nazir’s defence was that his story was drawn from childhood memories in rural Trinidad, written via speech-to-text on an Android phone due to chronic health conditions—which explains the unusual syntax critics flagged as machine-made. The Commonwealth Foundation reviewed the writing, choosing not to rely on flawed AI detection software but instead asked writers to provide drafts, story outlines, manuscripts and other evidence of their creative process. However, even though the Caribbean writer was vilified within days, the exoneration took weeks and has probably only reached a fraction of the audience.

This is not an isolated incident of a quick pile on, and slow exoneration. This has already occurred before with Mia Ballard’s Shy Girl, originally self-published in 2025 which built real grassroots success on social media and sales, and scored a traditional publishing deal off the back of that success. When the New York Times covered the story and offered the proof of AI detection software finding the novel AI-generated, Hachette cancelled the US release and discontinued UK sales—without any independent review process, no chance for the author to contest, and no transparency about methodology. Ballard’s defence that the editor she hired may have used AI editing software was reported and discarded. Ballard is now suing Hachette so any clarity or exoneration may be years in the coming.

The issue is not the accuracy of AI detection software, but instead the perception of what good and bad writing is based on a white perspective. In both these instances the authors in questions were non-white and while I’m not suggesting that Ballard’s claims of AI generative writing were based on her being black, because this was only identified in secondary reporting, I do believe that there is a bias within the writing world that is emulated by AI.

A study by Common Sense Media examining the use of generative AI in US schools found that “Black teens are more than twice as likely as White or Latino teens to say that teachers flagged their schoolwork as being created by generative AI when it was not (20% vs. 7% and 10%, respectively).” In another study researchers ran essays written by Chinese students for the Test of English as a Foreign Language, or TOEFL, through seven widely-used detectors and the tools incorrectly labelled more than half of them as AI generated. They also ran a sample of essays written by US eight graders who were native English speakers and these essays were identified as human-crafted. Meanwhile Denise Zubizarreta, who is a Latina, neurodivergent writer was flagged for “hyperbolic,” “repetitive” language—traits she ties to her cultural voice and dyslexic/AuDHD processing.

The fact is detectors don’t detect AI. They detect deviation from a narrow, implicitly white, implicitly able-bodied, implicitly native-English baseline of “normal” prose—and punish anyone who writes outside it, AI or not. When we consider that “between 1950 and 2000, 97 per cent of authors published by the literary giant Random House were white and that these are the books that were used to train AI generative software and detection software. Is it any wonder who gets flagged?

While both these case studies are exploring the US publishing scene, things are not any better in Australia. According to the Australian Publishing Industry Workforce Survey our industry is largely white, including a high percentage who identify as British; less than ten percent identify with an Asian culture, and less than 11% with a European (non-British) heritage. Fewer than 1% of Australian publishing industry professionals identify as First Nations.

It is within this white-washed landscape that “normal prose” is treated as suspect. What this reveals is that publishers and educational institutions need to be aware of the inherent biases within this new AI landscape and protect those most vulnerable to these accusations. As an educator I am recommending we implement workshops to teach students about ethical and unethical uses of AI, have students sign a contract of academic integrity, and that teachers compile student work portfolio samples to compare against when suspecting plagiarism.

And publishers need to take heed and have a disclosed, contestable review process before cancelling a contract on a single vendor’s score—not a 24-hour reaction to press coverage. Prizes and platforms should treat a detector’s output as a lead to investigate, not a verdict to publish. And most importantly readers and journalists should stop treating “the AI checker said so” as proof of anything, the same way no one would accept a polygraph result as courtroom evidence.

Until detection tools are audited and accountable, “AI slop” panic will keep finding the same targets—authors who already had the least institutional protection.

Amra Pajalic is an award-winning author of the Seka Torlak Mystery historical crime series which follows a journalist seeking justice in the aftermath of genocide—and the first three books are out now. www.amrapajalic.com