PDL and TVM-FFI

Intro This post is about the bubbles in you see in your Nsys traces and a couple interesting methods (tvm-ffi and Nvidia's Programmatic Dependent Launch) that aim to close those gaps. TVM-FFI The Nvidia docs strongly suggest compiling your CuTe kernel with tvm-ffi because of its ability to reduce launch overhead. The couple images below show an nsys trace with (top) and without (bottom) tvm-ffi.…