Gated Delta Net Attention: A Deep Dive into the Linear Attention Mechanism Powering Qwen3.5
A deep dive into the Gated Delta Net Attention mechanism, a linear attention technique powering Qwen3.5.
Pramodith Dissects
Pramodith Dissects is a blog covering AI, Tech, Startups and other musings.
Latest posts
Speculative Decoding Tutorial
A tutorial on implementing speculative decoding, an inference optimization technique for LLMs, using PyTorch and Hugging Face Transformers.
The Math Behind Online Softmax
Understanding the mathematical principles behind online softmax, an optimization technique used in Flash Attention to efficiently compute softmax in chunks.
The One Big Beautiful Blog on Group Relative Policy Optimization (GRPO)
A step-by-step tutorial to code up your own GRPO Trainer.
Do LLMs recognize Medical Definitions?
Figure: Do LLMs recognize Medical Definitions? There’s been a never-ending debate about whether LLMs understand the data they process, so much so that people have started to debate what understanding something actually means. While some argue that LLMs are mere statistical machines that capture patterns of when and where different words are used, others claim that LLMs do indeed have a sense ...