Hi anon, I'm subho, not a passionate AI engineer, please don't call me by that name. I'm rather a free soul, who likes to do epic shit in AI research. ...
A while ago I read a great post on percisely.xyz/gemm about writing fast GEMM kernels on Apple GPUs. The author talked about an undocumented Metal instruction called simdgroup_async_copy , used the intrinsic __metal_simdgroup_async_copy_2d to move tiles from device memory into threadgroup memory, and reported a counterintuitive finding. A single processor doing the load was much faster thanā¦
Mixture-of-Experts (MoE) routing is one of the most latency-sensitive operations in modern LLMs. Every forward pass computes a routing score matrix, selects the top-K experts, and softmax-normalises the weights before dispatching tokens. At inference scale this happens millions of times per second. Shaving microseconds here matters. This post walks through two implementations of a fused GEMM +ā¦
Modern deep learning applications use convolution almost all the time, and guess what convolution is the heart of image processing. From adding gaussian blur and edge convolution to identifying temporal relation among frames in a T2V model, convolution is just everywhere which presents us the opportunity to optimize it for good. In this worklog, we shall iteratively optimize 3D Square convolutionā¦
For those who are a bit new to ML, you might find this post to be a little difficult to comprehend but I shall try my best to make you clear till the end of this post. As the title suggests, we might be discussing about Preconditioned stochastic gradient descent and uncover some very interesting intuitions behind these optimizers and how they converge. So to start, we shall first scratch someā¦
We all needs attention in our lives, don't we? But its surprisingly difficult to compute the exponential terms which helps us taking the best plausible solution. It goes like this, Attention ( Q , K , V ) = softmax ( Q K T d k ) V The main issue here is Q K T multiplication, which perhaps has the complexity of O( n 2 d ). What's n and d? Itsā¦
In the rapidly evolving landscape of AI-assisted coding, Cursor AI has emerged as a groundbreaking tool that's transforming how developers write code. One of its most impressive features is "Speculative Edits" - a novel approach that dramatically improves code generation speed while maintaining high accuracy. What are Speculative Edits? It is a technique that anticipates low-entropy actionsā¦