Google Made Long Context 5x Cheaper

Google's TurboQuant compresses the KV cache by ~5x with minimal quality loss - what this means for open-source LLM inference.