Involutions: The License to Update in O(1)
Why Zobrist hashing updates a chessboard in constant time, and the property that makes incremental computation possible.
Software Bits Newsletter
Short stories on software design and performance.
Live Last read · last published · next check
Latest posts
Sparsity: The License to Skip
Why ignoring most of your neural network is the key to efficiency
Domain Transformations: The Art of Finding Easier Spaces
Why logarithms prevent underflow, why Fourier speeds up convolutions, and how choosing the right space makes hard problems tractable
Beyond Snowflake: A Problem-Solver’s Guide to Distributed ID Generation
How to discover solutions, not just memorize them.
Linearity: Why Batching Works
And the property that makes neural network training computationally tractable
Commutativity: Why Transformers Need Positional Encodings
And other consequences of order not mattering
The One Property That Makes FlashAttention Possible
What computing an average teaches us about the most important optimization in modern ML
PyTorch's compilation and matrix multiplication.
Or a case of missed optimizations
strcmp vs strncmp
Or why it's useful to know the size.
Faster reservoir sampling.
Or taking advantage of better algorithms.