Gram Newton-Schulz: A Fast, Hardware-Aware Newton-Schulz Algorithm for Muon

Muon is becoming the optimizer of choice for training state-of-the-art language models like Kimi K2 Thinking 1 and GLM-5. 2 Compared to AdamW, Muon needs fewer optimizer steps to reach a given loss, but each step is more expensive. This overhead is due to Muon’s Newton-Schulz orthogonalization procedure, a cubic time matrix operation not present in older optimizers. Figure 1: AdamW vs. Muon: Wall…