LLM inference is nearly deterministic. We use this to audit providers
Adam Karvonen, Daniel Reuter, Roy Rinberg, Luke Marks, Adrià Garriga-Alonso, Keri Warr · arXiv (paper link) · Github
Adam Karvonen
Adam Karvonen's personal website
Latest posts
Frontier AI Models Still Fail at Basic Physical Tasks: A Manufacturing Case Study
Dario Amodei, CEO of Anthropic, recently worried about a world where only 30% of jobs become automated, leading to class tensions between the automated and non-automated. Instead, he predicts that nearly all jobs will be automated simultaneously, putting everyone “in the same boat.” However, based on my experience spanning AI research (including first author papers at COLM / NeurIPS and attending…
Using an LLM perplexity filter to detect weight exfiltration
A recent area of focus has been securing AI model weights. If the weights are located in a data center and an adversary wants to obtain model weights, the weights have to leave physically (such as a hard drive going out the front door) or through the data center’s internet connection. If the facility has perfect physical security, then the weights have to leave through the internet connection.…
Evaluating Sparse Autoencoders with Board Games
This blog post discusses a collaborative research paper on sparse autoencoders (SAEs), specifically focusing on SAE evaluations and a new training method we call p-annealing. As the first author, I primarily contributed to the evaluation portion of our work. The views expressed here are my own and do not necessarily reflect the perspectives of my co-authors. You can access our full paper here.
An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability
Sparse Autoencoders (SAEs) have recently become popular for interpretability of machine learning models (although sparse dictionary learning has been around since 1997). Machine learning models and LLMs are becoming more powerful and useful, but they are still black boxes, and we don’t understand how they do the things that they are capable of. It seems like it would be useful if we could…
Manipulating Chess-GPT’s World Model
Manipulating Chess-GPT’s World Model
Chess-GPT’s Internal World Model
A Chess-GPT Linear Emergent World Representation