Tom Adamczewski's blog
The key tension in evals is now: how to make tasks that are difficult for AI, but actually fair? My default expectation in 2026 when I see a coding eval with low scores is that the tasks turn out to be impossible.
I predicted future AI models would someday learn to cheat on SWE-bench. They were already doing it.
Epoch AI is releasing a public registry of optimized Docker images for SWE-bench. This allows us to run SWE-bench Verified in 62 minutes on a single GitHub actions VM.
This report examines how to interpret AI timeline forecasts from AI experts. I consider how distribution fitting and aggregation methods influence results, and provide recommendations for how survey results should be analysed and presented in the future.
I made a mount to lift my laptop off the surface of my desk: Previous Next I operate my laptop with its lid closed, using only an external monitor. The downside is that the closed laptop is a useless rectangle that takes up a lot of precious desk space. This type of product would have solved 80% of the problem: However, even more space could be saved by removing the laptop from the desk entirely.…
I really don’t like cron jobs and crontab: crontab has a horrible syntax (it’s from 1975…) logging the output of jobs needs to be specified manually viewing logs is inconvenient (even just for checking whether a job ran or not!) cron jobs run in a minimal environment that’s inconvenient to modify
Many people think that lower child mortality causes fertility to decline. One prominent theory for this relationship, as described by Our World in Data 1 , is that “infant survival reduces the parents’ demand for children” 2 . (Infants are children under 1 years old). In this article, I want to look at how we can precisify that theory, and what magnitude the effect could possibly take. What…
Summary1: The likelihood function implied by an estimate bbb with standard deviation σ\sigmaσ is the probability density function (PDF) of a N(b,σ2)\mathcal{N}(b,\sigma^2)N(b,σ2). Though this might sound intuitive, it’s actually a special case. If we don’t firmly grasp that it’s an exception, it can be confusing. Thanks to Gavin Leech and Ben West for feedback on a previous versions of this post.
In the UK, many internet service providers (ISPs) block Sci-Hub . However, a simple proxy is enough to circumvent this (you don’t even need a VPN). Routing requests through a suitable 1 proxy lets you open Sci-Hub in your regular browser as if it weren’t blocked 2 . Routing all your traffic through a proxy may come with privacy and security concerns, and will slow your connection a bit. We want to…
Summary: I have tried many types of masks and respirators during the 2020 pandemic. My recommendation is to use ‘elastomeric’ respirators common in industry, and to either filter or completely block off their exhalation valve. The result is a comfortable respirator that I believe offers a high level of protection against airborne diseases to myself and others. I am not an infectious disease…