A 2,500-Layer Neural Network That Turned Out to Be MD5: What This Teaches About Debugging
Jane Street, one of the world’s most selective quantitative trading firms, published a mechanistic interpretability puzzle a few weeks ago. They hand-crafted a neural network with approximately 2,500 linear layers, integer weights, and released it to the public with a question: What function does this network compute? The answer: MD5. A cryptographic hash algorithm from 1992, implemented entirely as matrix multiplications and ReLU functions. What matters isn’t the answer. It’s the path the winner took to reach it. Because that path is, without exaggeration, a manual for debugging opaque systems that applies far beyond machine learning. ...