👋 Hi, I’m Hamza Elshafie I'm a Machine Learning Engineer passionate about everything related to deep learning and GPU programming lately focusing more on ...
Introduction Modern ML workloads depend heavily on custom GPU kernels. Even when a model is expressed as clean tensor operations, the performance almost always comes from specialised implementations underneath. Good examples of this are the many different attention mechanisms, GEMMs across different precisions, and MoE-style grouped GEMMs, which have become a fairly common architectural choice in…
🚧 Work in progress. Please reach out on LinkedIn if you spot any mistakes. Introduction Matrix multiplication sits at the core of modern deep learning. Whether it is transformers, CNNs, or even simple MLPs, everything eventually reduces to GEMM. GPUs are built to run this operation at scale, and libraries like cuBLAS set the performance bar with kernels tuned down to the last instruction. In this…
Large Language Models (LLMs) have been a critical point in the advancement of the field of AI, arguably the greatest technological innovation in the last decade (In NLP). These models, however, come with a sheer amount of model size, weights casually in the billions. As such, it’s become the default to run inference in the cloud, where serving providers have datacentres and GPU clusters that can…
Large language models (LLMs) are trained in highly parallel, compute-bound workloads, but serving them is very different: inference is memory-bound and sequential. Optimising inference is critical because no one will use a chatbot that lags behind typing or a tool that takes minutes to respond. On the business side, squeezing more out of each GPU directly reduces costs and maximises ROI. A key…