Tile Scheduling

Motivating the issue A matrix multiply is embarrassingly parallel. Each output tile can be computed independently. There are no data dependencies, no synchronization, and no ordering constraints. This is a kernel that can run at about 1.05x the speed of CuBLAS at M=N=K=8192 . Yet, when we naively order our tiles, don't use a persistent kernel, and don't use TMA multicast, our performance…