Leaderboard: every quant we measured

QuantDiff runs a diff between each popular GGUF download (from unsloth, bartowski, mradermacher and others) and the original full-precision model: how far the next-token probabilities drift (KL divergence), how often the top token still matches, whether tool calls still work, accuracy on fixed test sets, and the speed on a 12GB RTX 5070. Every run publishes its exact settings and raw logs. Just want an answer? Use the quant picker.

4models

31quants diffed

31runs, raw logs public

≤32Bscope

Jump to a best pick: Qwen3.5 4B · 8GB Qwen3.5 4B · 12GB Granite 4.2 8B · 8GB Granite 4.2 8B · 12GB Qwen3.5 9B · 8GB Qwen3.5 9B · 12GB Ornith 1.5 9B · 8GB Ornith 1.5 9B · 12GB

All measured quants

Model	Quant	Size	Fits	KLD ↓	Same top	Tool calls	GSM8K	MMLU	ARC-C	PP t/s	Gen t/s
Qwen3.5 4B	Q8_0 unsloth	4.17 GB	8GB+	0.0019	98.3%	86.5%	93.3 88–96	72.0 ≈ 65–78	96.0 ≈ 92–98	8442	114
Qwen3.5 4B	Q6_K bartowski	3.54 GB	8GB+	0.0037	97.1%	86.5%	93.3 88–96	73.5 67–79	96.0 ≈ 92–98	7188	130
Qwen3.5 4B	UD-Q4_K_XL unsloth	2.71 GB	8GB+	0.0220	93.3%	85.0%	91.3 ≈ 86–95	71.0 ≈ 64–77	96.0 ≈ 92–98	7928	151
Qwen3.5 4B	Q4_K_M bartowski	2.81 GB	8GB+	0.0258	92.8%	86.5%	90.0 ≈ 84–94	73.0 ≈ 66–79	95.3 ≈ 91–98	7774	152
Qwen3.5 4B	Q4_K_M unsloth	2.55 GB	8GB+	0.0308	92.4%	87.0%	92.0 ≈ 87–95	72.0 ≈ 65–78	96.7 92–99	7917	156
Qwen3.5 4B	i1-Q4_K_M mradermacher	2.52 GB	8GB+	0.0339	91.8%	89.5%	91.3 ≈ 86–95	72.0 ≈ 65–78	94.7 ≈ 90–97	7785	158
Qwen3.5 4B	IQ3_M bartowski	2.31 GB	8GB+	0.1002	86.3%	87.5%	89.3 ≈ 83–93	70.5 ≈ 64–76	96.0 ≈ 92–98	8197	161
Qwen3.5 4B	UD-Q2_K_XL unsloth	1.81 GB	8GB+	0.2425	78.6%	76.0%	78.0 ▼ below 71–84	67.5 ≈ 61–74	90.0 ≈ 84–94	6685	187
Granite 4.2 8B	Q8_0 bartowski	8.70 GB	12GB+	0.0082	97.8%	61.0%	88.7 ≈ 83–93	69.5 ≈ 63–75	88.7 ≈ 83–93	4574	65
Granite 4.2 8B	Q6_K bartowski	7.00 GB	12GB+	0.0136	96.2%	60.5%	90.7 85–94	67.5 ≈ 61–74	88.7 ≈ 83–93	3679	78
Granite 4.2 8B	Q5_K_M mradermacher	5.82 GB	12GB+	0.0579	92.1%	62.5%	90.7 85–94	68.0 ≈ 61–74	90.0 84–94	4220	92
Granite 4.2 8B	Q4_K_M bartowski	5.16 GB	8GB+	0.0729	91.0%	62.5%	88.7 ≈ 83–93	70.5 64–76	88.7 ≈ 83–93	4236	102
Granite 4.2 8B	IQ4_XS bartowski	4.60 GB	8GB+	0.0846	89.5%	62.5%	88.7 ≈ 83–93	69.5 ≈ 63–75	88.7 ≈ 83–93	4808	113
Granite 4.2 8B	i1-Q4_K_M mradermacher	4.98 GB	8GB+	0.0856	90.2%	60.5%	88.7 ≈ 83–93	68.5 ≈ 62–75	86.7 ≈ 80–91	4248	105
Granite 4.2 8B	IQ3_M bartowski	3.95 GB	8GB+	0.2233	82.9%	60.5%	85.3 ≈ 79–90	68.5 ≈ 62–75	87.3 ≈ 81–92	4402	120
Granite 4.2 8B	Q2_K bartowski	3.33 GB	8GB+	0.4598	75.0%	61.0%	85.3 ≈ 79–90	63.5 ≈ 57–70	84.0 ≈ 77–89	3315	146
Qwen3.5 9B	Q8_0 unsloth	8.87 GB	12GB+	0.0025	98.4%	91.5%	92.7 ≈ 87–96	78.0 ≈ 72–83	96.0 ≈ 92–98	4985	66
Qwen3.5 9B	Q6_K unsloth	6.95 GB	12GB+	0.0043	97.5%	92.5%	90.7 ≈ 85–94	78.0 ≈ 72–83	96.0 ≈ 92–98	4010	82
Qwen3.5 9B	Q5_K_M bartowski	6.62 GB	8GB+	0.0093	96.4%	92.5%	92.0 ≈ 87–95	78.0 ≈ 72–83	96.0 ≈ 92–98	4687	88
Qwen3.5 9B	UD-Q4_K_XL unsloth	5.56 GB	8GB+	0.0206	94.4%	90.5%	90.7 ≈ 85–94	78.0 ≈ 72–83	96.7 92–99	4719	97
Qwen3.5 9B	Q4_K_M bartowski	5.75 GB	8GB+	0.0219	94.0%	92.0%	93.3 88–96	79.0 ≈ 73–84	96.0 ≈ 92–98	4652	98
Qwen3.5 9B	IQ4_XS bartowski	5.12 GB	8GB+	0.0332	92.9%	89.5%	91.3 ≈ 86–95	76.5 ≈ 70–82	94.7 ≈ 90–97	5358	111
Qwen3.5 9B	i1-IQ4_XS mradermacher	4.72 GB	8GB+	0.0379	92.4%	89.5%	92.0 ≈ 87–95	76.0 ≈ 70–81	95.3 ≈ 91–98	5327	114
Qwen3.5 9B	Q4_K_M unsloth	5.29 GB	8GB+	0.0462	92.4%	91.5%	90.7 ≈ 85–94	79.5 73–85	96.7 92–99	4700	101
Qwen3.5 9B	i1-Q4_K_M mradermacher	5.24 GB	8GB+	0.0481	92.0%	92.0%	91.3 ≈ 86–95	78.0 ≈ 72–83	96.7 92–99	4707	101
Qwen3.5 9B	IQ3_M bartowski	4.64 GB	8GB+	0.0899	88.0%	88.0%	92.0 ≈ 87–95	74.5 ≈ 68–80	93.3 ≈ 88–96	4903	112
Qwen3.5 9B	UD-Q2_K_XL unsloth	3.84 GB	8GB+	0.2071	81.4%	82.0%	85.3 ≈ 79–90	72.5 ≈ 66–78	94.7 ≈ 90–97	3932	133
Ornith 1.5 9B	Q4_K_M ornith-ai	5.38 GB	8GB+	—	—	87.5%	88.7 ≈ 83–93	80.0 74–85	95.3 ≈ 91–98	4503	99
Ornith 1.5 9B	Q5_K_M ornith-ai	6.19 GB	8GB+	—	—	89.0%	90.7 85–94	78.0 ≈ 72–83	96.7 92–99	4589	90
Ornith 1.5 9B	Q6_K ornith-ai	7.04 GB	12GB+	—	—	89.0%	89.3 ≈ 83–93	78.5 ≈ 72–84	96.7 92–99	3937	83
Ornith 1.5 9B	Q8_0 ornith-ai	9.11 GB	12GB+	—	—	88.5%	89.3 ≈ 83–93	75.5 ≈ 69–81	96.7 92–99	4828	65

Accuracy cells: % correct, with the 95% confidence interval (Wilson) underneath. ≈ = statistically tied with the best quant of the same model (intervals overlap). ▼ below = clearly worse than that model's best quant (intervals do not overlap). Ties are not ranked: sorting by an accuracy column puts tied quants together and orders them by KLD. How accuracy is measured.

KLD = the diff: mean KL divergence of the quant's next-token distribution vs the BF16 model on the fixed QuantDiff text set (lower is better; under 0.01 is near-lossless). Same top = how often the quant's most likely next token matches the original's. Speed measured with llama-bench on an RTX 5070 12GB. GSM8K / MMLU / ARC-C = % correct on fixed 150 / 200 / 150-item subsets (lm-evaluation-harness, temperature 0, thinking off), measured with the same GGUF files on Ghost (CUDA, NVIDIA GeForce RTX 5070 12 GB (64 GB RAM)), minipro (Metal, Apple M2 Pro (Mac mini), 12-core CPU, 19-core GPU, 32 GB unified memory); differences under ~5 points are noise. "Fits" estimates file + 8192-token f16 KV cache + 0.6GB buffers, with 0.5GB kept free. Full methodology.