Systems for LLM RL
Foray into the systems challenges and approaches for LLM RL
Datta's Blog
Visual, math-backed explanations of LLM systems, attention, fine-tuning, GPU kernels, and the engineering details behind modern deep learning.
Latest posts
Reinforcement Learning for LLMs
A brief introduction to reinforcement learning for LLMs
The MathemaTricks behind FlashAttention
Fast and memory efficient exact attention
The lore behind LoRA
LoRA imagined from the ground up
Exploring the Mixture of Experts
An intuitive build up to Mixture of Experts