62 GGUF quants from 5 uploaders, measured by KL divergence against the BF16 reference using ~250,000 tokens of coding, chat, tool calling, science, non-Latin scripts, and long documents. Full methodology.
unsloth/gemma-4-E4B-it-GGUF (20 quants)
bartowski/google_gemma-4-E4B-it-GGUF (22 quants)
lmstudio-community/gemma-4-E4B-it-GGUF (3 quants)
ggml-org/gemma-4-E4B-it-GGUF (2 quants)
mradermacher/gemma-4-E4B-it-i1-GGUF (15 quants)
The best quant at each size. If it’s not in this table, a smaller file with lower KL exists.
Gemma 4 E4B is a small Mixture of Experts model. It quantizes extremely well. UD-Q8_K_XL has KL 0.003 and 98.7% top-1, unsloth Q8_0 has KL 0.003 and 98.6%. This is the best quantization tolerance of any model tested so far.
Q8_0 differs across uploaders. unsloth’s Q8_0 (8.2 GB, KL 0.003) is larger but measurably better than bartowski/ggml-org (8.0 GB, KL 0.006).
unsloth dominates the frontier: 13 of 17 points.
lmstudio-community never appears on the frontier. lmstudio-community and ggml-org Q4_K_M score KL 0.207/0.208 vs bartowski’s 0.072. Avoid them.
The entire model fits in 3.5-8.6 GB. Even UD-IQ2_M at 3.5 GB retains 78.8% top-1.
At Q8_0, all categories are essentially 0 (KL 0.001-0.006). The per-category spread only becomes visible below Q5. Long documents degrade fastest.
Inference: TextGen + patched llama.cpp (logprob extraction from prompt)
Reference: BF16 GGUF by unsloth
Dataset: ~250,000 tokens across 6 categories (coding, general chat, tool calling, science, non-Latin scripts, long documents)
Input format: full OpenAI-compatible messages rendered through the model’s Jinja2 chat template
Metric: KL divergence, computed token-by-token between reference and quantized top-40 log-probability distributions
Please do not share the plots and tables in this post publicly. Instead, share the post URL. These measurements are expensive to produce and more subscribers means more models benchmarked.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.