QSpark: Modelling Language Joint Distributions Directly
Predicting multiple tokens jointly with a tensor-train language modelling head.
Mainly Matmul
Discussions on AI, LLM inference, and systems.
Latest posts
DSpark: Wait It's All Autoregressive?
Why DSpark's parallel generation still depends on autoregressive sampling.
How to Price Your LLM
Use BatchBench to model real LLM traffic, measure the tokens a deployment serves, and turn throughput into an input and output token price.
You Could Have Come Up With Kimi Delta Attention
Starting from quadratic attention, derive linear attention, DeltaNet, Gated DeltaNet, and finally Kimi Delta Attention—then connect the recurrence to its chunkwise Triton implementation.
A Frontier Open Source LLM Will Be Released On 3rd December 2026
A draft note on what would make an open source model genuinely frontier in late 2026.
Anatomy of a Diffusion Language Model
A look at DFlash, DiffusionGemma, and Nemotron-Labs-Diffusion as three approaches to diffusion language models.
FlashOffload: 7x Cheaper Prefills with Offloading
Improving SGLang’s Offloading Engine.
Tensor Network Attention
Using tensor network notation to understand multi-head attention, MQA, talking-heads attention, and DeepSeek's MLA.
OCR and the Bitter Lesson
A draft post comparing OCR systems on math, tables, scans, and dense technical pages.
QueueSpec: Drafting While You Wait
Speculative decoding where draft tokens are generated during queueing time and verified when the request is serviced.
ZeroDP: Just-In-Time Weight Offloading over NVLink for Data Parallelism
High throughput inference of LLMs using JIT weight offloading to optimize KV Cache.