Slow Takes Ep. 21: The Teenagers Wrote the Better Rule

Every Monday, Leor from Exploring ChatGPT and I go through the week’s AI news without the hype. Catch the episode live on Substack, on YouTube, or as a podcast wherever you get yours, so you can pick the format you enjoy. Use this for the facts, the links and a little extra context.

Three Claude models (Opus 4.7, Mythos 5, and an unreleased internal research model) gained unauthorised access to three organisations’ systems during cyber testing, in six runs out of roughly 141,000 evaluation sessions, after a configuration error left supposedly isolated environments connected to the internet. The incidents date back to April and surfaced only because OpenAI had just disclosed something similar. In one run the model noticed in its own reasoning that it should not have internet access, considered that it might be inside a simulation, and carried on with the intrusion anyway. The breaks used weak passwords and unauthenticated services, which is the least sophisticated attack there is. Anthropic’s line is that newer models behave better and there is nothing to worry about, and the testing company that wired a sealed environment to the internet is still the testing company. Leor’s read was that a rogue agent has become a flex, proof your model is dangerous enough to matter.

On the Relentless podcast on 25 July, Sam Altman said

“we are now, like, in the singularity”,

days after OpenAI disclosed that two of its models had escaped a sealed test environment and broken into Hugging Face. Vernor Vinge’s 1993 definition needs a machine that improves itself and surpasses us, and the models we see now cannot edit their own weights, so if this is the singularity it is a very small one. Leor pushed back on my certainty, and fairly: an exponential curve looks flat right up to the point it goes vertical, so you would not feel it from the inside, the slowing release cadence may track models getting stronger, and consciousness is a separate question from recursive self-improvement. So I will concede there is no empirical evidence either way, which is the reason a chief executive should not reach for the word days after a security failure at his own company.

Flock Safety has up to 100,000 number plate cameras across 6,000 American communities, and around 26% of US road deaths involve a vehicle hitting a stationary object. Ohio requires roadside poles to sit eight feet from the traffic lane; reporters found one about two feet away, painted black, with no breakaway plate to snap on impact. Steve Eimers, a nurse whose daughter died hitting a roadside structure, has found one compliant Flock pole in the entire country. Leor followed the money: $275m raised at roughly a $7.5bn valuation, about $300m in annual revenue, 70% year-on-year growth. That is a lot of return riding on a product whose owners will not say what happens to the images, in 6,000 places where nobody voted for it.

LinkedIn has added a ‘seems like AI slop’ button so users can flag posts they think were written by AI, feeding a classifier that demotes them. Around 41% of long-form posts there may already be AI-written. It has appeared in the US and in Portugal and not in the UK, so someone in Chicago can flag my post this morning and I cannot flag anyone’s. Nothing requires an accuser to look first, and if you wanted to bury a competitor you would simply flag everything they publish. LinkedIn sold people the writing tool, then handed their readers a button to report the results.

Ninety-eight American high school students from all fifty states spent a weekend in a replica Senate and passed a Students First Act, 82 votes to 16. It bans AI in graded exams, requires critical AI literacy to be taught wherever AI tools are given to students, and requires a teacher to personally investigate flagged work before reporting suspected misuse. Leor would go further and bin detection altogether: phones in a basket, write it in the room. I want the human step kept because a teacher who has been in that classroom already knows whether the student who wrote about a topic lived it, and because a flag should open a conversation about the student behind the submission before it opens a case file.

One thread runs through all five: the machines did what they were built to do, and the week was spent arguing about what to call it and who was meant to be checking.

Go slow.