The kernel that wasn't there: profiling my way to real-time streaming Zonos

Or: what 7% taught me before I wrote a single kernel. I set out to write a fused Mamba2 decode kernel for Zonos. The plan, in writing, said 2 to 3x on the per-frame decode step. The first thing profiling told me was that the kernel I was about to write would buy me about 7%, and that the part of it everyone treats as the bottleneck was already sitting at the memory roofline. So I didn't write it.…