Jamie Simon — ML researcher and writer. Personal site with blog posts on machine learning theory, math, physics, and science.
There’s a question that’s ringing out all over the machine learning world these days: what the hell counts as an “explanation” of a complex system? If you listen for a while to conversations in the labs of practitioners, the forums of mechanistic interpretability, the ivory halls of theory, or even the labs of AI-savvy neuroscientists or biologists, you’ll start hearing this question thrumming…
I learned something beautiful today and just wanted to share it. Obviously, this blog is super eclectic, with tons of posts on science and math and art and life, with no particular order. Maybe someday I’ll organize it. This a pretty basic calculus trick. In fact, I wouldn’t be surprised if I’d come across it in high school and just forgot it. But today it came up in my research, and I had to…
The language of polynomials is beautiful and well-developed: millenia of work have furnished a rich, deep theory that spans analysis and algebra, with basics we teach to children and high reaches still under development. The language of neural network features has not yet been written, but it promises to be beautiful: empirics show that hidden features can represent complex concepts and form…
I love classic folk and country songs. I listened to so many at camp growing up, songs like “Rocky Top” or “Country Roads” or John Prine’s “Paradise” that tug at the heartstrings and make you yearn for the simple things. A really good folk song gives you the feeling that it’s been around forever. It’s like a pebble in a creek, worn smooth from the tumbling of being sung again and again. In 2018, I…
This is a technical note on kernel theory. Venture in at your own risk! My collaborators and I recently put out a paper called Predicting kernel regression learning curves from raw data statistics in which, among other things, we show how to map from the data PCA stats to the eigenfunctions and eigenvalues of a broad swath of rotation-invariant kernels. It’s a surprising result, both…
In 2019, I spent six months in Gothenburg, Sweden at the Chalmers Institute of Technology. I was working in the lab of Per Delsing doing nanofabrication in my gap year before grad school, and it was not going well. Have you ever wondered how computer chips are made? A chip is basically a block of silicon covered in an immense tangle of thin wires and pieces of doped silicon. These wires are…
My brother’s girlfriend’s dad, Tom Zimmer, had the craziest job out of high school I’ve ever heard of. He shipped up to a wildlife preserve in Wisconsin run by the International Crane Foundation and there found his job was, among other things, to artificially inseminate cranes. They wouldn’t breed in captivity, so he had to go to their pens — the male first, then the female — to collect and…
I find it beautiful and fascinating how a melody can be changed or enhanced by an artistic choice of the underlying chords. As a study in chord choice, I spent an afternoon reharmonizing Ode to Joy. Here’s my finished product. Your browser does not support the audio element. Even though it’s really short, this was a lot of fun to work on. It was really satisfying to see how adding sevenths to…
Two friends asked me to play Mendelssohn’s Wedding March from A Midsummer Night’s Dream at their ceremony. After a day learning the piece and a few hours in GarageBand, I sent them this and told them they’d be getting the Muppets version. Your browser does not support the audio element. Congratulations to Isaac Harris and Sophia Mock, thanks to Bennett Witcher for some good Muppet noises, and…
sometimes I feel there is aaaaccddggklnnnnnooprrttuuvwyy never anything truly new aaaccddeeefgiiklmmnoooprsssttu only old things taken apart accdeeeeefgiimmnnorrsstuuvwy and the pieces rearranged acefgiikllmmnnnooosstttuuvwyy and put together in new configurations aacdeeekllmmrssvyy in an ever-changing dance of parts deeeikllmmoorsstttuuwyy