What Counts as AI-Written?

3 Comments Post your own or leave a trackback: Trackback URL

1. rrhersh says:

   August 15, 2026 at 7:51 am

   I finally found a use case for an LLM. I have a book manuscript in editing. It is a word history book on baseball vocabulary. It has many citations, mostly from newspapers, which in this case are primary sources. I have a naive belief that citation style should serve the book, rather than the other way around, so I put these citations in the text in parentheses. My editor, however, is an acolyte of Chicago, and asked me to provide an alphabetical list of all these newspaper citations, presumably to be stuck in the back of the book. What purpose this will serve is not clear to me, but fine.

   Making the list would be a total pain, so instead I put the text through ChatGPT with instructions to copy everything inside parentheses and make an alphabetical list.

   The results are like the work product of an enthusiastic but not very bright intern. I didn’t ask it to discriminate between newspaper citations and other parentheticals, because I had no expectation it would be able to do this. So fine. I had to break the text up, as I certainly am not paying for a more robust version, asking it to do the same thing as the last batch. Sometimes this worked, but sometimes it had forgotten what it had done and I had to repeat myself. Each batch came out formatted differently, just because. And while it was mostly alphabetized correctly, there were mistakes. This one surprised me, as I would have thought this was something an LLM would be good at.

   All in all, it was better than doing it by hand. But it also is a very narrow use case.

   Reply
2. rrhersh says:

   August 15, 2026 at 7:56 am

   As for the quiz, if any significant part of the final text is produced by AI, I am out regardless of whether the ostensible author reviewed it. This is not on moralistic grounds, but because AI writing is mediocre. Given how it is trained, this is practically by definition. If the ostensible author reads the AI text and concludes that it is at least as good as anything he could produce, he is probably right. But this is a condemnation of his writing skills, not praise of the AI’s. I am slightly more lenient of AI outlines. Outlining seems to be an obstacle for otherwise competent writers, and the AI work product is likely to be good enough.

   Reply
   • rhollick says:

     August 15, 2026 at 11:20 am

     But it seems to me that the detection of “a significant part” remains a problem. “AI writing is mediocre” is a tough yardstick to apply. Lots of non-AI-writing is mediocre. But I suppose just avoiding mediocre writing will work!

     This quiz isn’t the best way to assess opinions I know, but I thought it was the right sort of approach. The more we know about what people specifically think the more we’ll be able to see the way towards regulation. Just what the route to regulation is is far from clear though.

     I suppose it is possible that your newspaper citation sorting may be done more efficiently by future versions(?).

     Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.