Gemma 4 26B A4B GGUF Quality Benchmark: unsloth, bartowski, lmstudio-community, ggml-org, mradermacher, mudler compared

80 GGUF quants from 6 uploaders, measured by KL divergence against the BF16 reference using ~250,000 tokens of coding, chat, tool calling, science, non-Latin scripts, and long documents. Full methodology.

• unsloth/gemma-4-26B-A4B-it-GGUF (21 quants)

• bartowski/google_gemma-4-26B-A4B-it-GGUF (26 quants)

• lmstudio-community/gemma-4-26B-A4B-it-GGUF (3 quants)

• ggml-org/gemma-4-26B-A4B-it-GGUF (2 quants)

• mradermacher/gemma-4-26B-A4B-it-i1-GGUF (21 quants)

• mudler/gemma-4-26B-A4B-it-APEX-GGUF (7 quants)

The best quant at each size. If it’s not in this table, a smaller file with lower KL exists.

• This is a Mixture of Experts model with 4B active parameters out of 26B total. It quantizes much worse than the dense 31B. Q8_0 already shows KL 0.544 and 77.9% top-1, compared to 0.163 and 92.0% for the dense Gemma 4 31B. That’s 3.3x worse KL at the same quant level.

• The Pareto frontier is dominated by unsloth UD quants. Only bartowski Q6_K_L and two IQ2_XXS entries (bartowski, mradermacher) at the very bottom break the streak. ggml-org, lmstudio-community, and mudler never appear on the frontier (except Q8_0 where everyone is the same).

• ggml-org and lmstudio-community Q4_K_M score KL 2.126/2.116 vs bartowski’s 1.093 at the same quant type. Avoid them.

• mudler’s APEX quants never appear on the frontier.

• unsloth’s MXFP4_MOE (16.6 GB, KL 0.963) is beaten by their own UD-Q4_K_XL (17.1 GB, KL 0.747). The MoE-specific quant format doesn’t help.

Long documents hit KL 1.055 at Q8_0. For comparison, the dense 31B has 0.466 at Q8_0. All categories degrade roughly in proportion, with science and tool calling the most resilient throughout.

• Inference: TextGen + patched llama.cpp (logprob extraction from prompt)

• Reference: BF16 GGUF by unsloth

• Dataset: ~250,000 tokens across 6 categories (coding, general chat, tool calling, science, non-Latin scripts, long documents)

• Input format: full OpenAI-compatible messages rendered through the model’s Jinja2 chat template

• Metric: KL divergence, computed token-by-token between reference and quantized top-40 log-probability distributions

Full methodology

Please do not share the plots and tables in this post publicly. Instead, share the post URL. These measurements are expensive to produce and more subscribers means more models benchmarked.