72 GGUF quants from 4 uploaders, measured by KL divergence against the BF16 reference using ~250,000 tokens of coding, chat, tool calling, science, non-Latin scripts, and long documents. Full methodology.
unsloth/Qwen3.5-9B-GGUF (21 quants)
bartowski/Qwen_Qwen3.5-9B-GGUF (24 quants)
lmstudio-community/Qwen3.5-9B-GGUF (3 quants)
mradermacher/Qwen3.5-9B-i1-GGUF (24 quants)
The best quant at each size. If it’s not in this table, a smaller file with lower KL exists.
Qwen 3.5 9B quantizes better than its larger siblings. Q8_0 has KL 0.066, compared to 0.120 for Qwen 3.5 27B and 0.121 for Qwen 3.5 35B A3B.
No single uploader dominates. bartowski wins the mid-range (5-8 GB), mradermacher owns below 4 GB, and unsloth’s UD quants fill the gaps.
lmstudio-community never appears on the frontier. Avoid them.
Even at 3.4 GB (IQ2_M), the model retains 79.3% top-1. Below that, quality drops fast: IQ1_S at 2.4 GB is barely functional at 53.7% top-1.
Long documents dominate the quality loss: UD-Q8_K_XL scores KL 0.001 on coding but 0.288 on long documents. Q8_0 is even worse at 0.367 for long documents. All other categories stay at or below 0.028 at Q8_0.
Inference: TextGen + patched llama.cpp (logprob extraction from prompt)
Reference: BF16 GGUF by unsloth
Dataset: ~250,000 tokens across 6 categories (coding, general chat, tool calling, science, non-Latin scripts, long documents)
Input format: full OpenAI-compatible messages rendered through the model’s Jinja2 chat template
Metric: KL divergence, computed token-by-token between reference and quantized top-40 log-probability distributions
Please do not share the plots and tables in this post publicly. Instead, share the post URL. These measurements are expensive to produce and more subscribers means more models benchmarked.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.