Best Ornith 1.5 9B quant for your GPU
4 quantized files of ornith-ai/Ornith-1.5-9B (9.2B): accuracy, tool calls and speed measured; KLD vs the BF16 original pending.
ornith-ai/Ornith-1.5-9B (MIT). Per its model card, built on Qwen3.5 with continued pre-training and RL post-training; same hybrid architecture as Qwen3.5 9B (32 layers, 8 full attention + 24 linear attention). Quants from the official ornith-ai GGUF repo, which only has Q4_K_M, Q5_K_M, Q6_K and Q8_0 (no Q2/Q3). Tested with thinking disabled.
KLD pending. KL divergence vs the BF16 reference is queued on our Apple M2 Pro test Mac and will be added here when measured. Until then, picks for this model are ranked by measured accuracy (GSM8K + MMLU + ARC-C, 500 items, 95% CI) with tool calls as tie-break.
Quality vs file size
Left: KLD pending. Right: accuracy on all 500 items (GSM8K + MMLU + ARC-C). Error bars = 95% intervals. Download PNG to share.
Not sure which fits? Use the quant picker →
Recommendation by VRAM
"Best" = highest measured accuracy (500 items pooled) among files that fit fully on the GPU (file + 8192-token KV cache), tool-call score as tie-breaker. KLD pending.
8GB GPU
- File size
- 6.19 GB
- KLD
- pending
- Accuracy (500 items)
- 87.4% (84.2–90.0)
- Tool calls
- 89.0%
- Accuracy
- tied with best
- Generation
- 90 t/s full GPU
Fastest that fits: Q4_K_M (ornith-ai, 99 t/s, acc 87.2%, KLD pending)
12GB GPU
- File size
- 6.19 GB
- KLD
- pending
- Accuracy (500 items)
- 87.4% (84.2–90.0)
- Tool calls
- 89.0%
- Accuracy
- tied with best
- Generation
- 90 t/s full GPU
Fastest that fits: Q4_K_M (ornith-ai, 99 t/s, acc 87.2%, KLD pending)
16GB GPU
- File size
- 6.19 GB
- KLD
- pending
- Accuracy (500 items)
- 87.4% (84.2–90.0)
- Tool calls
- 89.0%
- Accuracy
- tied with best
- Generation
- 90 t/s full GPU
Fastest that fits: Q4_K_M (ornith-ai, 99 t/s, acc 87.2%, KLD pending)
All Ornith 1.5 9B quants
| Quant | Publisher | Size | Fits | KLD mean | Same top | Tool calls | Gen t/s | Run |
|---|---|---|---|---|---|---|---|---|
| Q4_K_M | ornith-ai | 5.38 GB | 8GB+ | pending | — | 87.5% | 99 | settings & logs |
| Q5_K_M | ornith-ai | 6.19 GB | 8GB+ | pending | — | 89.0% | 90 | settings & logs |
| Q6_K | ornith-ai | 7.04 GB | 12GB+ | pending | — | 89.0% | 83 | settings & logs |
| Q8_0 | ornith-ai | 9.11 GB | 12GB+ | pending | — | 88.5% | 65 | settings & logs |
Accuracy cells: % correct, with the 95% confidence interval (Wilson) underneath. ≈ = statistically tied with the best quant of the same model (intervals overlap). ▼ below = clearly worse than that model's best quant (intervals do not overlap). Ties are not ranked: sorting by an accuracy column puts tied quants together and orders them by KLD. How accuracy is measured.
GB = GiB (1024³ bytes). KLD, tool calls and speeds from Ghost (RTX 5070 12GB, CUDA, Ryzen 7 5800X, 64GB DDR4). On a different GPU, expect generation speed to scale roughly with memory bandwidth. Task accuracy (GSM8K / MMLU / ARC-C, % correct on fixed subsets of 150 / 200 / 150 items, frozen suite quantdiff-lmeval-v1) was measured with the same GGUF files and llama.cpp build on Ghost (CUDA, NVIDIA GeForce RTX 5070 12 GB (64 GB RAM)), minipro (Metal, Apple M2 Pro (Mac mini), 12-core CPU, 19-core GPU, 32 GB unified memory). Accuracy depends on the weights, not on the GPU, but small numeric differences between backends can flip individual answers. With these sample sizes, differences under about 5 points are within noise (± shown on run pages). Methodology.
GPU too small? Rent one.
Run a 32B model at Q6/Q8, or a 70B, on a cloud GPU for an hour or two instead of buying hardware.
Cloud GPU pods with 24-80GB cards (RTX 4090 / 5090 / A100 / H100) for big quants or BF16 reference runs.
Rent on RunPodGPU marketplace for renting a 24-48GB card for an evening of testing.
Rent on Vast.aiReferral links: QuantDiff may earn credit if you sign up. Referral partners have no say in scores or rankings.