MXFP8 GEMM: Up to 99% of cuBLAS performance using CUDA + PTX
I recently did a deep-dive on writing GEMM kernels with just CUDA + PTX for Ampere, Hopper, and Blackwell GPUs, culminating in a MXFP8 GEMM kernel which achieves up to 99% of cuBLAS (torch._scaled_mm) depending on the problem shape - see microbenchmarks below, measured with: B200 GPU, 1000W power CUDA 13.0 PyTorch version: 2.11.0+cu130 K-major input operands, with float32 output