Who Holds the Power to Watch?

This is an opinion piece. I’m not telling you what to think. I’m asking you to think.

This is the longer version of this article (C1 for B2 readers). It has more research, more data and more voices from the classroom. If you prefer shorter sentences and simpler vocabulary, there is a B1-B2 version of this article here with the same argument, the same classroom quotes and the same question at the end.

Here’s the thing about this week: I sat in a near-one-to-one session with a managed-services engineer (I’ll call him Marco), and by the end of it I realised I agreed with him more than was comfortable.

Marco works in security. He watches customer devices for a living. And he thinks the German privacy debate has completely lost the plot.

“The privacy thing is something we’re discussing too much, too often,” he told me. “Because if I’m chatting with my girlfriend, it’s privacy, but on the other side, there’s nothing very important in it for a third party not knowing me.” Then the sharper version: “We are wasting money, resources, time, energy on some things that is totally the wrong way, and we could invest all these into better solutions.”

Now, one man in one room is one strong opinion, and hardly a national consensus. Nobody pushed back in the moment, partly because it was nearly just the two of us, and partly (I’ll be honest) because I kept nodding. But it seeded something I’ve been chewing on all week, across seven other groups, in an industry that watches networks and installs cameras and writes the very white papers that shape European surveillance law.

The topic was simple to state and horrible to answer: to make AI safe, must we accept being watched — and who decides? That question came from the headlines. In June, the US government reached into a private AI company and switched off its most capable model overnight, worldwide, in the name of safety. Europe is still fighting over whether to scan everyone’s private messages to catch a handful of criminals. The uncomfortable logic underneath both is this: to guarantee a powerful tool is never misused, someone has to watch how everyone uses it.

So the whole week, I kept asking myself which question we’re actually arguing about. And I don’t think it’s the one on the poster. Let me show you why.

Key Vocabulary — Six Weeks That Made It Concrete

• dual-use | technology that can be used both for good and for harm | doppelter Verwendungszweck

• export controls | government rules restricting who abroad may access a technology | Exportkontrollen

• to gatekeep | to control who is allowed access to something | den Zugang kontrollieren

• key escrow | a system where the government keeps a copy of every encryption key | Schlüsselhinterlegung

• informational self-determination | your right to decide who knows what about you | informationelle Selbstbestimmung

• proportionate | reasonable in scale compared with the aim; not more intrusive than necessary | verhältnismäßig

• a carve-out | a special exemption from a rule for one particular group | Ausnahmeregelung

Before the opinions, the facts. Because this stopped being abstract in June.

On 12 June 2026, at 5:21pm Eastern Time, the US Commerce Department ordered Anthropic to suspend foreign access to its two most capable models, Fable 5 and Mythos 5, after a reported jailbreak of Fable’s safeguards. Because it couldn’t screen users by nationality in real time, Anthropic switched both off for everyone on Earth. Access returned in stages: around a hundred trusted US critical-infrastructure organisations on 26–27 June, then fully on 30 June–1 July, once guardrails were agreed. Anthropic disputed the premise, calling the technique only “minor, previously known vulnerabilities” that also existed in rivals like GPT-5.5.

Whatever you think of the risk, notice the instrument. A government used export controls, a tool built for missiles and microchips, as an ad-hoc approval gate for a piece of software the whole world was already using. That is the precedent that should make you sit up, whichever side you’re on. Frontier AI is genuinely dual-use: the same model that drafts a vaccine protocol can lower the barrier to a bioweapon. So the instinct to gatekeep it is understandable. The question is what you build to do it.

Europe is running the other half of the experiment. “Chat Control,” the proposal to scan private messages for child-abuse material, is watching-everyone-to-catch-the-few made law, and the European Commission’s own figures are the strongest case against it. Detection tools err between 13% and 20% of the time. Nearly 50% of the reports reaching Germany’s Federal Criminal Police (BKA) are criminally irrelevant. And the confirmed hit rate? 0.000002735% of everything scanned. Read that number again. You would search a whole population’s messages to find almost nobody, while the people writing the rules quietly discuss exempting professional-secrecy and official accounts. That last part is my charge, not established fact: but if the class designing the surveillance writes itself a carve-out, that tells you how much they trust it.

We have been here before. In 1993 the Clinton administration wanted a key escrow system, the “Clipper chip,” handing the state a copy of the key to every secure conversation. Cryptographers found a fatal flaw, business revolted, and by 1996 it was dead while strong encryption spread worldwide anyway. Gatekeeping dual-use software has a long history of failing, and of doing real damage on the way down.

And Germany, more than anyone, knows why the reflex to resist runs deep. In 1983 the Federal Constitutional Court struck down a routine census and wrote a line that still anchors European law: under modern data processing, there is “no such thing as a trivial datum” (kein belangloses Datum). From that ruling comes informational self-determination: your right to decide who knows what about you. Whether a “little surveillance for safety” is proportionate is not a new question here. It’s a national scar.

Key Vocabulary — Where the Watching Actually Reaches

• to surveil | to watch people systematically, usually without their consent | überwachen

• function creep | the gradual expansion of a tool or power beyond its original purpose | schleichende Zweckausweitung

• regulatory capture | when rules end up serving the established companies that helped write them | regulatorische Vereinnahmung

• an incumbent | an established player that already dominates a market | etablierter Platzhirsch

• a chilling effect | when people stop doing something legal because they fear being watched | Abschreckungseffekt

• a compute threshold | a level of computing power above which extra rules apply | Rechenleistungs-Schwellenwert

Here’s what makes this more than a developers’ problem. To police powerful AI at scale, you have to watch usage at scale. And once you build the machinery to surveil usage, it rarely stays pointed where it started. That’s function creep, and history says it’s the rule. Police crime-mapping systems grow into protest-attendance mapping; the apparatus outlives its excuse.

The proposed levers all have this property. Licensing the largest training runs above a certain compute threshold. “Know your customer” rules for cloud providers. Mandatory logging of how models are used. Each is defensible on its own. Together they describe a system that observes everyone in order to catch someone.

And the same move hands enormous advantage to the biggest players. Marc Andreessen, the Netscape founder turned venture capitalist, makes this point bluntly: rules written with incumbents serve incumbents. Regulatory capture is just the ordinary result when compliance costs that only giants can afford become the price of entry; no conspiracy required. He and others put the existential-risk scenarios near zero, and Sam Altman reportedly dismissed a rival’s safety posture as “fear-based marketing.” When a government can switch off a critical tool overnight, a builder reads that as a chilling effect on everyone downstream.

The counter-voice is just as serious, and it comes from inside the industry. Dario Amodei, Anthropic’s CEO, says plainly: “AI-enabled authoritarianism terrifies me.” He warns that the Chinese state has “the clearest path to the AI-enabled totalitarian nightmare,” a “perfect tyrant’s toolkit” of total surveillance, autonomous drones and personalised propaganda. His case for building oversight first: if democracies don’t, authoritarians will define what “monitoring” means for everyone else.

That’s not hypothetical. According to Carnegie’s global index (the figures date to 2019, so read them as a floor), Chinese firms were already supplying AI-surveillance technology to 63 countries, with facial recognition running in 64 and “safe city” platforms in 56. The export bundles a governance model in with the hardware.

And it reaches ordinary desks faster than you’d think. In December 2023, France’s data regulator fined Amazon’s French logistics arm €32 million for monitoring warehouse staff so closely that one indicator (nicknamed the “stow machine gun”) flagged any item scanned in under 1.25 seconds. This wasn’t even frontier AI: a scanner, a stopwatch, and 6,200 people measured to the second, every reading logged. The watching always arrives at the shop floor eventually.

Key Vocabulary — What Happened in the Room

• to steelman | to present the strongest, fairest version of a view you disagree with | das stärkste Gegenargument bilden

• a base rate | how common something is across the whole population, before you test for it | Grundrate, Basisrate

• a false positive | a test result that flags a problem when there isn’t one | Fehlalarm, falsch positiv

• purpose limitation | the rule that data may be used only for the reason it was collected | Zweckbindung

• a works council | an elected body representing employees in a German company | Betriebsrat

• consent | agreement given freely and knowingly | Einwilligung

• liability | legal responsibility for harm caused | Haftung

This is where the week earned its keep. I taught it across groups who don’t just hold opinions about surveillance — several build it, operate it, or defend against it for a living. The richest material came from the people who’d actually done the thing.

Start with Stefan, an account lead at a network-and-security firm, because he did something the whole debate needs: he executed the strict-security position entirely inside the privacy campaigners’ guardrails. His rowing club had a vandalism problem, so he installed cameras. Ten of them. Motion detection only. Footage auto-deleted after 72 hours. Board-only access behind multi-factor authentication. He documented for every member why the cameras were there and what happened to the footage. The objections “evaporated after a ten-minute talk.” He also cited an AI-camera pilot elsewhere in Germany, where “if there is a fight on the local place, KI [German for AI] detects the fight and the camera will call the police.” He offered it as a small, local safety win.

Then he told the other half, the honest one: his own mayor blocked cameras on a village sports pitch after roughly half a million euros of vandalism, “because it brings too much problems, too much discussions.” A purpose-limited safety measure, killed on privacy grounds. Both things are true at once, and he lived both.

Against that, the cleanest argument for control came from Andreas, at a legal-tech company, as a rebuttal to me. I’d argued that determined bad actors route around any rule; they used Google before AI existed. He didn’t deny it, he reframed the scale. The really dangerous people, he said, are “hopefully just a handful.” But “if you allow it with AI, then hundreds of people and more are getting this information and will reproduce the stuff.” That’s the strongest steelman of the security case all week: AI multiplies the number of capable villains. I don’t have a clean answer to it. That’s rather the point.

Others reached the numbers instinctively. Nadine, at the same firm, reconstructed the whole Chat Control problem before I gave her the figures, reasoning from the base rate: “even a tiny error rate can produce thousands or millions of incorrect alerts, because the number of lawful messages vastly exceeds the number of illegal ones.” That’s the mathematics of the false positive, worked out from first principles, in her second language, under real pressure. Honestly, better than half the newspaper columns on the subject.

Then the pivot the whole debate turns on. Viktor, an engineer at the same firm, cut through all three positions in four words: “Who controls the government?” Then he sharpened it into something physical: “government can take control of a self-driving car; if you do something wrong in their opinion, they can drive you against the wall.” Melodramatic? Maybe. But it names the real fear precisely: the question is who is watching, and what else they can already reach.

Not everyone feared the state most. Katrin, a one-to-one client in corporate AI governance, was more anxious about what she couldn’t see: “We cannot control what is happening to the information we put in the system.” Then she confessed the modern contradiction beautifully, having fed her own finances to an AI for analysis: “the system knows a lot about me: where I work, how much I earn, where I invest. It has a quite clear profile of my person. Maybe I’m too open.” Her company still can’t use AI officially, because they haven’t cleared it with their works council. The blocker is consent and control; the tool itself works fine.

Jonas, a recruiter at a managed-services firm, drew his line in exactly the right place. He runs candidates’ role and career history through an AI screening tool, but never “address, phone data, date of birth.” It can build a professional profile, “but not on the specific personal data of that person. That’s where I would draw the line.” On why Germans feel this so hard, he shrugged: “I don’t know if that comes from the Stasi era.” (The lawyers would tell you the missing term is purpose limitation (Zweckbindung): data collected for one reason shouldn’t quietly serve another. It’s also where much of the liability lives.)

And to keep me honest, two voices argued the German privacy reflex is overdone. Petra, at the same firm, put the moderate version: the culture is “too strong; in the past it works also, and there wasn’t so many regulations.” When I pushed back with “that was Stasi Germany,” she held her ground: “And it works also in West Germany.” She sat in the same room as the function-creep warnings, and neither side was silly.

The privacy defenders weren’t paranoid either. Bettina named the mechanism: “When I accept small, what is the next step of this? The Germans had a hard past with Stasi — they don’t want to reach this level again.” And Sabine held the reasonable middle so many people live in: “Some monitoring is ok if it helps that people are safe. I accept that little monitoring, if my personal information is kept private.”

Three genuine positions. None of them the obvious villain. That’s the room. Now here’s where I land.

Key Vocabulary — Who Holds the Power to Watch

• misallocation | putting effort or resources where they do the least good | Fehlallokation

• the thin end of the wedge | a small first step that opens the way to something much larger | der Anfang vom Ende

• informed consent | agreement given with full understanding of what you’re agreeing to | informierte Einwilligung

• to decentralise | to spread power away from a single central authority | dezentralisieren

• to ratchet | to increase step by step in a way that’s hard to reverse | schrittweise verstärken

• disproportionate | too large or severe compared with the aim | unverhältnismäßig

Fair warning: this is the part where I stop reporting and start leaning. I lean, but I don’t pretend to be certain, and by the end you’ll see exactly where I run out of road.

1. The debate is misallocated, and yes, a bit hypocritical. Huge energy, near-zero return. When a government proposes scanning the entire population’s messages to confirm a rounding error’s worth of real crime, you’ve built theatre and called it protection. On the numbers alone it is wildly disproportionate. And “one rule for the population and another for the EU elites” is the tell. This is a charge about misallocation, about where the effort goes. Please don’t hear “privacy is worthless.” Hear: we are spending our fear in the wrong place.

2. The wrong question. This is the one place I’ll spend it, because it’s the whole argument: it’s not how much we’re watched — it’s who holds the power to watch. Fixating on the amount is how you end up arguing about a signposted camera at a boat club while missing the machine that actually scares you.

3. Stefan’s cameras prove bounded watching works. Overt, signposted, purpose-clear, delete-by-default surveillance, run by a private club on its own property, to deter vandals and catch the rare bad actor — that is legitimate. “If you are aware of the surveillance and WHY it is there, then you should behave.” The reflexive “but my privacy” objection to that is overblown. A club that tells you where every camera is and wipes the tape in 72 hours is a caretaker with a hard drive, and calling it the Stasi is how you lose the argument that actually matters.

4. The power answer to the slippery slope. Now, the strongest objection to me, Bettina’s, is the thin end of the wedge: the friendly boat-club camera is exactly how the apparatus begins, then retention creeps, then it’s shared with the police, then it ratchets into something you can’t reverse. I take that seriously, because history is on her side. But my answer runs through location. A boat club can’t become Big Brother; it hasn’t the reach. A state can. So the safeguard against function creep is to decentralise the watching, keep it with clubs, companies and individuals, and starve the state of it. “Privately owned versus state owned: less power to the state, more to the people.”

Here’s the tension I won’t smooth over, though. Private doesn’t automatically mean safe. Meta and the data brokers are among the largest watchers alive, and “they argue privacy when it suits them and then give everything to Instagram” cuts both ways: it accuses the hypocrite and the harvester. My honest distinction is this: private power is escapable. You can leave Instagram. You cannot leave the state. That asymmetry is the whole reason I’d rather the cameras sat in a thousand private hands than one public one — not because corporations are angels.

5. Try and fail, don’t regulate-and-block. “You cannot protect everyone without blocking all progress.” The precautionary instinct feels responsible, but it has a price nobody puts on the invoice: the tool never built, the incentive to try that quietly dies. Better to try and fail and try again than to discuss, regulate and block for three years. If that sounds familiar, it should: it’s “progress over perfection” wearing a policy suit. I’ve spent nearly twenty years speaking German badly and confidently, in front of clients who could have corrected me and mostly, kindly, chose not to. Had I waited until my dative case behaved itself before opening my mouth, I’d still be sitting there silent in 2026, gender-checking every noun. Perfection doesn’t exist in language, and it doesn’t exist in governance either. You build the thing, you watch it wobble, you fix it.

6. The consumer’s deal. For you, the individual, the bargain with these tools is informed consent: understand how a tool uses your data, accept the ambiguity you can’t remove, and that acceptance is what hands you control over how, and how much, you use it. Disagree with the data practices? Don’t use it. Don’t care? That’s a legitimate choice too. The choice is yours. (This, incidentally, is the same principle I teach for thinking, only pointed at your data instead of your ideas.)

7. The counterweight I can’t dismiss, and the bridge back. Risk is permanent. Harvested personal data really can be assembled into targeted attacks: the social-engineering, build-a-profile-and-aim-it move that Cambridge Analytica made famous. That doesn’t go away because I find the debate overdone. But that, on the other hand, is also life. People should have a choice, and most of us are magnificent hypocrites about it anyway — invoking privacy when it’s convenient, then handing our whole financial life to a chatbot because it drew a nice pie chart. Katrin admitted it. So would I. So, if you’re honest, would you. And to the privacy defender who carries the Stasi memory in their bones: your fear of function creep is legitimate, and it is precisely a fear of state power. We want the same thing. I’m not arguing you out of it. I’m arguing for where to point it: keep the signposted camera, and make sure the state never becomes the one holding all of them.

And here, at the exact moment you’d want me to tie a bow on it, is where I stop, because the honest answer is a shrug.

The lesson’s real question was who should police the safe use of frontier AI in the future? Government? I don’t want too much government control. The developers? I don’t think the companies building the AI should write their own rules and dig a profit moat while they’re at it. The AI itself? I don’t want a world policed by AI. An external body? I’m not convinced one could keep up. That’s four candidates and I’ve rejected all four. I genuinely don’t know who should hold the reins.

And I’ll go further, because the person I’m least honest with here is usually me. I’m not even sure what I would truly give up to keep these tools. I love what AI has unlocked in my business. I see the enormous future and the dark side in the same glance. But the dark side has always existed, and most of us simply choose to ignore it. It’s there. Always lurking in the background. Always a risk. I use the tools anyway — and if you’re reading this on a device that knows where you are, so do you.

A companion piece runs alongside this one: the same three positions, rebuilt as a free English lesson at B1, B2 and C1: the assigned-position debate, the vocabulary, and the discussion questions, all scaffolded on The BEBB Method — The Agency Loop, my five-step framework for using AI without letting it replace your own thinking. It’s my own configuration of established best practice in human-AI collaboration, built in direct response to Gerlich (2025) on cognitive offloading, and informed by the “AI Sandwich” tradition (Ippolito, 2023) and “AI as Critic” scaffolding (Mollick, 2024). If you teach, you can run the whole thing. Read the companion lesson →

If this was useful, share it with a colleague who’s arguing about the wrong question.

Where do you stand? And what would you actually give up?

Share