Flash Attention from Scratch Part 4: Bank Conflicts & Swizzling

Intro In the last part, we used the instructions covered in Part 2 to construct our first kernel and reached nearly half the performance of the official implementation on the RTX 3090.