Re-learning Backprop From Five Lines of NumPy

When I started writing the flash attention series, embeddings were on my “I get it intuitively” list. A word becomes an N-dimensional vector, the geometry encodes meaning, cats are near dogs and not near typewriters. That was enough to write the forward pass. For GPU code, embeddings are floats with shape and the kernel doesn’t care what the floats mean.