New blog
My blog is currently at https://adam.scherl.is.
Adam Scherlis
Parameter Space: The Final Frontier
Latest posts
Two Percolation Puzzles
Take a very large chessboard (NxN, where N is huge). Remove some fraction 1-p of the squares at random, leaving a fraction p of them. Can you place a queen on the first row and then, via some sequence of Continue reading
GPT-175bee
Epistemic status: whimsical Bees: a new unit of measurement for ML model size Talking about modern ML models inevitably leads to a bunch of hard-to-intuit large numbers, especially when it comes to parameter count. To address this, Lawrence Chan and I propose Continue reading
How to export Android Chrome tabs to an HTML file in Linux (as of February 2023)
Let s say you have a few million tabs open in your mobile Chrome browser, because you never close anything, but now your browser is getting slow and laggy. You want to stick the URLs of those tabs somewhere for safekeeping Continue reading
Inner Misalignment in “Simulator” LLMs
As seen on Alignment Forum and LessWrong Alternate title: Somewhat Contra Scott On Simulators . Scott Alexander has a recent post up on large language models as simulators. I generally agree with Part I of the post, which advocates thinking about LLMs as Continue reading
Fun math facts about 2023
2023=7×172 Maybe that s not fun enough? Try this: 2023=211−52 Or better yet: 20233=(31176029+245568392)/(384321573) We can scientifically quantify how fun a math fact is, so we can rest assured that this is the funnest fact about 2023 ever discovered. But if it s not Continue reading
A hundredth of a bit of extra entropy
There are two ways to calculate the amount of information in one term of a continued fraction: These differ by about 0.0088 bits. It took me a while to figure out why they were different at all, and now I m Continue reading
An exploration of GPT-2’s embedding weights
I wrote this doc in December 2021, while working at Redwood Research. It summarizes a handful of observations about GPT-2 s weights mostly the embedding matrix, but also the LayerNorm gain parameters that I found while doing some open-ended Continue reading
A brainteaser for language models
I came up with the following puzzle the other day: The intended answer is in the form of a number. text-davinci-003 guesses my intended answer at 11.8% probability, which is the second-highest probability for any answer. (This is somewhat cherry-picked; small Continue reading
New Frontiers in Mojibake
Fun with mismatched encodings Mojibake is the garbled text that result from character-encoding errors. If you’ve seen text that looks like this — and I’m sure you have — then you’ve seen mojibake. (You should be seeing something like this: Continue reading