I recently watched Grant Sanderson's (3blue1brown) video about volume of high-dimensional spheres. He made a note that the high-dimensional space is also of peculiar interest to the ML field. I knew of the curse of dimensionality , but I wanted to dig deeper and here is what I found, the other side; the boon of dimensionality . I would recommend you watch Grant's video for this. I will try my best…
I recently participated in GPU Mode's NVFP4 Kernel Hackathon. The goal was to write CUDA kernels for Blackwell's new 4-bit floating point format (NVFP4) and get as close as possible to the hardware's speed of light. The competition ran on B200 GPUs, and there were four tasks in total: matrix-vector multiplication (GEMV), matrix-matrix multiplication (GEMM), gated GEMM (with SiLU activation), and…
import Figure from "@/components/Figure.astro"; Layouts are a powerful abstraction introduced in NVIDIA's CuTe library for making operations on complicated Tensor configurations a little bit easier to understand. My goal here is to provide a good taste of how operations on these layouts work and an example matrix-matrix multiplication kernel to show the value of these abstractions and its…
import Figure from "@/components/Figure.astro"; You might have heard of this algorithm during the RAG-mania of late. But not many sources on the web go into the inner workings and intuition building, which I would like to cover here. Hierarchical Navigable Small Worlds (HNSW) is a top-performing and popular approximate nearest neighbor search algorithm. To find the exact nearest neighbor given a…
Some of the systems I work with are highly customized environments, and often need support engineers to maintain them. A lot of automation exists, but sometimes they need to manually go into a VM and change things. This is normal , but with these manual tasks, mistakes are inevitable. One such case is a service that would only work if all the files and directories it manages are owned by a special…
One of the hardest parts of balancing load across many targets is keeping an accurate view of their load. We have to check all the targets; this is expensive. Also, the process of checking is not instantaneous, which leads to stale data which in turn causes request herding. What if we assign a request to a random target? This works surprisingly well but causes hotspots, which is not ideal. A…
Very early in my career I had the absolute delight of reading SICP ; this was my first introduction Lispy languages. I used Racket to go through the book at the time, and found it overall enjoyable to use. Although, I found the all the parentheses to be an acquired taste. Eventually could visually cut through the parentheses. Clojure was one other language I wanted to explore in this genre. Aside…
WARNING Contents of this post were published in 2020, while some concepts are still relevant, most of provided code might not work due to framework changes. Machine Learning models are getting bigger and expensive to compute. Embedded devices have restricted memory, computation power and battery. But we can optimize our model to run smoothly on these devices. By reducing the size of the model we…