Flash Attention 2 in CuteDSL: A Naive Kernel, Three Optimizations, and Where I Got Stuck

After writing Flash Attention 1 in plain CUDA, I wanted to take the next step and port Flash Attention 2 to CuteDSL , the Python front-end NVIDIA ships with CUTLASS 4. CuteDSL is new, under-documented, and surprisingly fun once you wrap your head around it. I do not write kernels every day, and I'm still building intuition for both GPU hardware and CuteDSL itself, so this project involved a lot of…