Kimi K3 is Changing the AI Playbook

The main takeaway from Kimi K3’s launch last week wasn’t that Chinese and open source models are closing in on frontier models. The takeaway also wasn’t that cheaper models mean more inference. GLM, Qwen, and DeepSeek already highlighted these developments and why we covered it a month ago. It’s that architectural and software efficiency can compensate for raw compute. This has important knock-on effects to the AI infra trade.

Kimi K3 is a huge model - 2.8 trillion parameters. Each parameter is a single number but as a simple example think about each of them as a book. K3 uses a sparse Mixture of Experts (MoE) approach. Think about grouping all the books into different categories where you have specialist librarians for each - K3 has 896 experts (sub-networks) but only activates less than 2% per token (just 16). Per token refers to the answer an LLM gives you which is just a string of tokens produced one at a time.

The older language models would run every parameter on every token - an incredibly resource-intensive task with some of today’s larger models. Sparse MoE has become more popular over the past few years and getting K3 to <2% of parameters while delivering performance has set a benchmark in this space - 2.8 trillion parameters, less than 2% activation, with a 1 million token context window in an open source model.

Kimi also demoed MiniTriton which is a Triton-like compiler K3 built from scratch. In plain language, you need a software layer that lets you run an AI model on a GPU - a compiler is one vital piece in that layer. This software layer is crucial and good vs bad software can result in multiples better performance for the same model on the same hardware. For Nvidia’s B200, software optimizations delivered 5x lower cost per token in just 2 months.

The software layer is what has helped Nvidia retain such a dominant position in GPUs. Over nearly two decades, Nvidia has built CUDA (languages, compilers, libraries of pre-optimized code) that makes it easy to get strong performance out of its chips - virtually every AI framework and engineer has built on top of it. Competing chips, whether from AMD or Chinese vendors, often have decent hardware on paper but lack this mature software stack, so the same model runs slower and with more engineering headaches.

If AI models can start to build viable compiler infrastructure from scratch (what K3 has partly shown with MiniTriton), then Nvidia's software moat starts to erode.