Google published TurboQuant on March 24th. Within 48 hours the paper had 575 points on Hacker News, Micron’s stock dropped $900 million, and TechCrunch compared it to Pied Piper’s algorithm from Silicon Valley.
One month later, the hype fog has cleared enough to answer the only questions that matter: Does it work? Can I use it today? And the one nobody wants to ask: Is it actually new?
What TurboQuant promises (30-second recap)
If you already read my previous article on the math, skip this section.
TurboQuant compresses the KV cache of LLMs — the working memory that grows with each generated token — by applying two transformations:
- Random rotation of the vector (redistributes values uniformly)
- Conversion to polar coordinates (angles fall into predictable ranges → quantization without overhead)
Google’s claimed result: 6x KV cache compression, 8x speedup in attention logits on H100, no degradation in benchmarks. No retraining. No calibration. Data-oblivious.
Sounds too good to be true. Let’s see what reality says.
The real ecosystem: what implementations exist
In one month, at least ten independent implementations have appeared. I classify them by maturity:
Tier 1: Functional with reproducible benchmarks
| Project | Language | Platform | Real compression | Status |
|---|---|---|---|---|
| turboquant-mlx | Python/Metal | Apple Silicon | 4.7x (3-bit V3) | Published benchmarks, M4 Max |
| turboquant-pytorch | Python | CUDA | 5x (K4/V2) | From-scratch, tests vs paper |
| turboquant (0xSero) | Python | CUDA | 3-bit K, 2-bit V | Triton kernels + vLLM |
Tier 2: Functional but no independent benchmarks
| Project | Language | Platform | Notes |
|---|---|---|---|
| SwiftLM | Swift | Apple Silicon | Native inference server, integrated TurboQuant KV cache |
| TurboVec (py-turboquant) | Rust + Python | CPU | Focused on vector search, not KV cache |
| turboquant_mlx | Python/Metal | Apple Silicon | 1-3 bit, asymmetric |
Tier 3: PRs in large projects
- llama.cpp: Discussion #20969 — CPU implementation with tests, MSE within 1% of paper
- vLLM: PR under review for native integration
The data point that matters: sharpner’s MLX implementation achieves 4.7x real compression on M4 Max with 3-bit Lloyd-Max. Not 6x as Google claims, but still a substantial reduction. In a 16K token context, KV cache drops from 4.2 GB to 897 MB.
The finding nobody expected: QJL doesn’t work
This is where it gets interesting.
TurboQuant has two phases: PolarQuant (rotation + polar + quantization) and QJL (error correction with 1-bit Johnson-Lindenstrauss). Google presents both as essential.
But multiple independent implementers have reached the same conclusion: QJL degrades performance in practice.
The turboquant-pytorch team measured it rigorously. Their finding: MSE-only (without QJL) consistently outperforms MSE+QJL in token matching — the metric that actually matters for text generation. The difference is enormous at low bits and still visible at 8 bits.
The second half of TurboQuant, which Google presents as error correction, introduces more errors than it fixes. 80% of the value is in PolarQuant alone.
This type of finding — components that work in the paper but not in production — is exactly what separates research from engineering. And it’s what makes independent implementations valuable.
The RaBitQ controversy: the elephant in the room
This is the point that TechCrunch coverage and Twitter threads don’t mention.
On March 31st, the RaBitQ team — a vector quantization method published in May 2024 — formally accused TurboQuant’s authors of three things:
1. Deliberate omission of methodological similarity. Both TurboQuant and RaBitQ apply random rotations (Johnson-Lindenstrauss transforms) before quantizing. That’s the core trick of both. But Google’s paper describes RaBitQ as “grid-based PQ” and omits that it uses rotation. TurboQuant’s second author knew RaBitQ perfectly — he asked for help debugging it in January 2025.
2. Misrepresentation of theoretical results. TurboQuant claims RaBitQ’s guarantees are “suboptimal, likely due to loose analysis.” But RaBitQ has a rigorous proof (Theorem 3.2) that it achieves the optimal asymptotic bounds established by Alon and Klartag (FOCS 2017). RaBitQ’s authors sent detailed corrections by email in May 2025. TurboQuant’s second author confirmed sharing them with all co-authors. The final paper corrected nothing.
3. Biased experimental setup. TurboQuant compared RaBitQ using a degraded Python translation, single-core, with multithreading disabled. TurboQuant ran on A100 GPU. RaBitQ has a public C++ multithreaded implementation. The speed comparison is, diplomatically speaking, unrepresentative.
RaBitQ’s team has filed a formal complaint with ICLR’s ethics committee and published a comment on OpenReview. Stanford NLP amplified the accusation.
Does this invalidate TurboQuant? No. PolarQuant works — benchmarks from independent implementations confirm it. But the paper has an academic integrity problem that Google Research hasn’t addressed publicly. And for a company wanting to be a reference in open research, silence is significant.
Try it today: three paths based on your hardware
Path 1: Apple Silicon with MLX (recommended for Mac)
git clone https://github.com/sharpner/turboquant-mlx
cd turboquant-mlx
pip install -r requirements.txt
# Benchmark with Llama 3.2 3B (4-bit weights)
python bench.py --model mlx-community/Llama-3.2-3B-Instruct-4bit \
--kv-bits 3 --method v3 --seq-len 8192
Expected results on M4 Pro (48 GB):
- KV cache compression: ~4.6x
- Throughput: ~98% of baseline without compression
- Perplexity: practically identical to FP16
Path 2: CUDA with PyTorch
git clone https://github.com/tonbistudio/turboquant-pytorch
cd turboquant-pytorch
pip install -r requirements.txt
# Generation test with Qwen2.5-3B
python tests/generation_test.py --bits 4 --model Qwen/Qwen2.5-3B-Instruct
Key finding from the team: K4/V2 (4 bits for keys, 2 for values) gives ~5x compression with acceptable quality. Asymmetric works better than symmetric because keys determine attention patterns.
Path 3: Native inference server in Swift
If you want an OpenAI-compatible server running TurboQuant on Apple Silicon without Python:
git clone https://github.com/SharpAI/SwiftLM
cd SwiftLM
swift build -c release
# Serve Gemma 4 26B with compressed KV cache
.build/release/SwiftLM serve --model mlx-community/gemma-4-26b-it-4bit \
--turbo-kv
SwiftLM reports 85 tok/s with Gemma-4-26B-4bit on M5 Pro. TurboQuant KV cache enables 100K token contexts that wouldn’t fit in memory without compression.
What you CAN’T do (yet)
Apply TurboQuant to Apple’s on-device model. The FoundationModels framework in iOS 26 / macOS 26 manages KV cache internally through LanguageModelSession. There’s no API to intercept, compress, or extend it. Apple’s model has 4K token context and TurboQuant can’t change that — at least not with public APIs.
Use it in production with vLLM. The PR exists but hasn’t been merged. 0xSero’s Triton kernels integration works but doesn’t have the testing level that a production framework demands.
Trust the paper’s vector search benchmarks. Given that the RaBitQ comparison uses a biased setup, the paper’s vector search numbers aren’t reliable. PolarQuant works; the paper’s comparative benchmarks, not so much.
The bigger pattern: coordination vs compression
TurboQuant is a case study of something happening more often in ML: the technique works, but the narrative around it is bigger than the technique.
What actually works is PolarQuant: random rotation + polar coordinates. It’s elegant, practical, and compresses KV cache 4-5x on real hardware. Not 6x as Google claims, not 8x as newspapers headlined, but enough to extend contexts 4x in the same memory.
What doesn’t work as well: QJL (error correction) and the narrative of total originality (RaBitQ was already doing random rotations in 2024).
And what’s unacceptable: a Google paper in ICLR that misrepresents previous work, uses biased benchmarks, and then stays silent when errors are pointed out. That’s not a technical error, it’s an institutional problem.
Next time a paper from a major lab promises 6x compression at no cost, do what independent implementers have done: grab the code, measure yourself, and be prepared to discover that reality lies somewhere between what’s promised and what’s useful. Which is usually, anyway, a pretty good place to be.
Sources:
- TurboQuant: Redefining AI Efficiency with Extreme Compression — Google Research Blog
- TurboQuant and RaBitQ: What the Public Story Gets Wrong — Jianyang Gao (RaBitQ author)
- TurboQuant paper (ICLR 2026) — OpenReview
- Hacker News discussion (575 points, 166 comments)
This article was originally written in Spanish and translated with the help of AI.