90 GGUF quants from 8 uploaders, measured by KL divergence against the BF16 reference using ~250,000 tokens of coding, chat, tool calling, science, non-Latin scripts, and long documents. Full methodology.
unsloth/Qwen3.5-35B-A3B-GGUF (23 quants)
bartowski/Qwen_Qwen3.5-35B-A3B-GGUF (27 quants)
lmstudio-community/Qwen3.5-35B-A3B-GGUF (3 quants)
ggml-org/Qwen3.5-35B-A3B-GGUF (1 quant)
mradermacher/Qwen3.5-35B-A3B-i1-GGUF (23 quants)
mudler/Qwen3.5-35B-A3B-APEX-GGUF (7 quants)
AesSedai/Qwen3.5-35B-A3B-GGUF (4 quants)
ubergarm/Qwen3.5-35B-A3B-GGUF (2 quants, ik_llama.cpp)
The best quant at each size. If it’s not in this table, a smaller file with lower KL exists.
This is a Mixture of Experts model with 3B active parameters out of 35B total. Unlike the Gemma 4 26B A4B, it quantizes well. Q8_0 has KL 0.121 and 95.4% top-1, close to the dense Qwen 3.5 27B (0.120, 97.0%). The Gemma 4 26B A4B has 0.544 and 77.9% at Q8_0.
Q8_0 is not on the Pareto frontier. UD-Q6_K_XL (32.1 GB, KL 0.112) is smaller and better.
unsloth dominates: 18 of 23 frontier points. The pattern is similar to the Gemma 4 26B A4B, but the quality loss is much smaller.
8 uploaders tested. AesSedai, mudler (APEX), ubergarm, ggml-org, and lmstudio-community never appear on the frontier.
Even at IQ2_XXS (9.9 GB), top-1 is still 83.6%.
Long documents degrade the fastest (KL 0.336 at UD-Q8_K_XL). Tool calling is the second worst (KL 0.152). Coding and science stay below 0.01 until Q4 levels.
Inference: TextGen + patched llama.cpp (logprob extraction from prompt)
Reference: BF16 GGUF by unsloth
Dataset: ~250,000 tokens across 6 categories (coding, general chat, tool calling, science, non-Latin scripts, long documents)
Input format: full OpenAI-compatible messages rendered through the model’s Jinja2 chat template
Metric: KL divergence, computed token-by-token between reference and quantized top-40 log-probability distributions
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.