Vulnerabilities and exploits: where are we headed?
Why AI makes vulnerability discovery defense-favoring (after a rough 2026-2027) but keeps exploitation offense-favoring. A follow-up to my Epoch AI piece.
Timothee Chauvin
Updates from https://tchauvin.com. Focus: AI safety and cybersecurity.
Live Last read · last published · next check
Latest posts
Cheaply detecting changes in LLM APIs
Links: [X thread] [LT paper] [B3IT paper] [tchauvin.com blog post]
24 theses on cybersecurity and AI
Getting to ninety-five isn’t so simple
[paper review] Top Score on the Wrong Exam: On Benchmarking in Machine Learning for Vulnerability Detection
[paper]
[paper review] Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risk of Language Models
[paper, website]
The hacker and the rationalist
Target audience: cybersecurity people, and AI / AI safety people.
Cybersecurity in AI: where progress is needed
Below are some areas in the field of cybersecurity in AI where I would like to see progress.
Preprint is out! eyeballvul: a future-proof benchmark for vulnerability detection in the wild
A previous blog post introduced the eyeballvul vulnerability detection benchmark.
[paper review] NYU CTF Dataset: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security
[paper]
Introducing the eyeballvul benchmark
Today I’m releasing eyeballvul, an open-source benchmark designed to enable the evaluation of SAST vulnerability detection tools, especially ones based on language models.