Learn CUTLASS the hard way - part 2!
Exploring Hopper Architecture on H100 and try to match PyTorch/cuBLAS performance for GEMMs using CUTLASS
Kapil Sharma
This is the website/technical blog of Kapil Sharma. I work at Meta on the Pytorch team.
Latest posts
Learn CUTLASS the hard way!
Walkthrough of optimization techniques for GEMMs from a naive fp32 kernel to CUTLASS bf16 kernel
Triton Kernels - Fused Softmax - 2
Worklog: Performance debugging Triton Kernel
Triton Kernels - Fused Softmax
Fused Softmax Triton kernel exploration
Triton Kernels - RMS Norm
RMS Normalization Triton kernel implementation for LLMs