Daniel Barabander
718
posts
@dbarabander
Joined March 2024
Pinned
Seems like a really good time to look into the law of distillation . . . we wrote the most complete guide here: x.com/dbarabander/st…
We can finally talk about it: We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company. We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.
Watermarking is an interesting way for Anthropic to build a moat. Only Anthropic has the secret key to verify whether its text outputs are AI generated. This turns text outputs from a complete "bearer asset" (no dependency on a party for authenticity) into something in which
If you want to learn how watermarking AI generated text works, I helped write this in-depth explanation while I was at Cravath. cravath.com/a/web/vvmaTCtn…
Anthropic says new Claude models will embed invisible watermarks in all generated text, everywhere Claude is offered. The watermark is part of the text, it isn't metadata: "it will travel with the text when it's copied and pasted elsewhere, and may persist through some editing."
Totally unexplored security paradigm - proof of physical presence. High value actions (authorizations to transfer money, change user roles, etc.) require proving you are physically in a pre-set location. Attacker cannot spoof being in that location halfway across the world.
Let the agents run. For the actions that really matter, shipping code, minting an admin key, moving money, make a person approve it and prove they were actually there. An agent can steal every password and key you have and still not be anywhere. Passwords copy endlessly. You're
