See what quantization changed.
QuantDiff runs a diff between each popular GGUF download (from unsloth, bartowski, mradermacher and others) and the original full-precision model. It checks how far the next-token probabilities drift (KL divergence), how often the top token still matches, whether tool calls still work, and what you gain in speed on a 12GB RTX 5070. Then it picks the file that changes the least for your GPU. Every run publishes its exact settings and raw logs.
Leaderboard
Download JSON / CSV| Model | Quant | Size | Fits | KLD ↓ | Same top | Tool calls | Gen t/s |
|---|---|---|---|---|---|---|---|
| Qwen3.5 4B | Q8_0 unsloth | 4.17 GB | 8GB+ | 0.0019 | 98.3% | 86.5% | 114 |
| Qwen3.5 4B | Q6_K bartowski | 3.54 GB | 8GB+ | 0.0037 | 97.1% | 86.5% | 130 |
| Qwen3.5 4B | UD-Q4_K_XL unsloth | 2.71 GB | 8GB+ | 0.0220 | 93.3% | 85.0% | 151 |
| Qwen3.5 4B | Q4_K_M bartowski | 2.81 GB | 8GB+ | 0.0258 | 92.8% | 86.5% | 152 |
| Qwen3.5 4B | Q4_K_M unsloth | 2.55 GB | 8GB+ | 0.0308 | 92.4% | 87.0% | 156 |
| Qwen3.5 4B | i1-Q4_K_M mradermacher | 2.52 GB | 8GB+ | 0.0339 | 91.8% | 89.5% | 158 |
| Qwen3.5 4B | IQ3_M bartowski | 2.31 GB | 8GB+ | 0.1002 | 86.3% | 87.5% | 161 |
| Qwen3.5 4B | UD-Q2_K_XL unsloth | 1.81 GB | 8GB+ | 0.2425 | 78.6% | 76.0% | 187 |
| Granite 4.2 8B | Q8_0 bartowski | 8.70 GB | 12GB+ | 0.0082 | 97.8% | 61.0% | 65 |
| Granite 4.2 8B | Q6_K bartowski | 7.00 GB | 12GB+ | 0.0136 | 96.2% | 60.5% | 78 |
| Granite 4.2 8B | Q5_K_M mradermacher | 5.82 GB | 12GB+ | 0.0579 | 92.1% | 62.5% | 92 |
| Granite 4.2 8B | Q4_K_M bartowski | 5.16 GB | 8GB+ | 0.0729 | 91.0% | 62.5% | 102 |
| Granite 4.2 8B | IQ4_XS bartowski | 4.60 GB | 8GB+ | 0.0846 | 89.5% | 62.5% | 113 |
| Granite 4.2 8B | i1-Q4_K_M mradermacher | 4.98 GB | 8GB+ | 0.0856 | 90.2% | 60.5% | 105 |
| Granite 4.2 8B | IQ3_M bartowski | 3.95 GB | 8GB+ | 0.2233 | 82.9% | 60.5% | 120 |
| Granite 4.2 8B | Q2_K bartowski | 3.33 GB | 8GB+ | 0.4598 | 75.0% | 61.0% | 146 |
| Qwen3.5 9B | Q8_0 unsloth | 8.87 GB | 12GB+ | 0.0025 | 98.4% | 91.5% | 66 |
| Qwen3.5 9B | Q6_K unsloth | 6.95 GB | 12GB+ | 0.0043 | 97.5% | 92.5% | 82 |
| Qwen3.5 9B | Q5_K_M bartowski | 6.62 GB | 8GB+ | 0.0093 | 96.4% | 92.5% | 88 |
| Qwen3.5 9B | UD-Q4_K_XL unsloth | 5.56 GB | 8GB+ | 0.0206 | 94.4% | 90.5% | 97 |
| Qwen3.5 9B | Q4_K_M bartowski | 5.75 GB | 8GB+ | 0.0219 | 94.0% | 92.0% | 98 |
| Qwen3.5 9B | IQ4_XS bartowski | 5.12 GB | 8GB+ | 0.0332 | 92.9% | 89.5% | 111 |
| Qwen3.5 9B | i1-IQ4_XS mradermacher | 4.72 GB | 8GB+ | 0.0379 | 92.4% | 89.5% | 114 |
| Qwen3.5 9B | Q4_K_M unsloth | 5.29 GB | 8GB+ | 0.0462 | 92.4% | 91.5% | 101 |
| Qwen3.5 9B | i1-Q4_K_M mradermacher | 5.24 GB | 8GB+ | 0.0481 | 92.0% | 92.0% | 101 |
| Qwen3.5 9B | IQ3_M bartowski | 4.64 GB | 8GB+ | 0.0899 | 88.0% | 88.0% | 112 |
| Qwen3.5 9B | UD-Q2_K_XL unsloth | 3.84 GB | 8GB+ | 0.2071 | 81.4% | 82.0% | 133 |
KLD = the diff: mean KL divergence of the quant's next-token distribution vs the BF16 model on the fixed QuantDiff text set (lower is better; under 0.01 is near-lossless). Same top = how often the quant's most likely next token matches the original's. Speed measured with llama-bench on an RTX 5070 12GB. "Fits" estimates file + 8192-token f16 KV cache + 0.6GB buffers, with 0.5GB kept free. Full methodology.
Best picks by model
- 8GB Q8_0 (unsloth, KLD 0.0019)
- 12GB Q8_0 (unsloth, KLD 0.0019)
- 16GB Q8_0 (unsloth, KLD 0.0019)
- 8GB Q4_K_M (bartowski, KLD 0.0729)
- 12GB Q8_0 (bartowski, KLD 0.0082)
- 16GB Q8_0 (bartowski, KLD 0.0082)
- 8GB Q5_K_M (bartowski, KLD 0.0093)
- 12GB Q8_0 (unsloth, KLD 0.0025)
- 16GB Q8_0 (unsloth, KLD 0.0025)
GPU too small? Rent one.
Run a 32B model at Q6/Q8, or a 70B, on a cloud GPU for an hour or two instead of buying hardware.
Per-second GPU pods. RTX 4090 / 5090 / A100 / H100 for big quants or BF16 reference runs.
Link coming soonMarketplace GPUs, often the cheapest way to try a 24-48GB card for an evening.
Link coming soon