Best Sage-A1.5 quant for a 16GB GPU
Short answer: i1-IQ3_M from mradermacher. Of the 2 Sage-A1.5 files we tested that fit fully in 16GB with a 8192-token context, it has the highest measured accuracy (88.0% on GSM8K + MMLU + ARC-C, 95% CI 84.9–90.6). Accuracy and speed only, no KLD: reference too large.
- File size
- 15.8 GB
- no KLD: reference too large
- Accuracy (500 items)
- 88.0% (84.9–90.6)
- 84.0%
- Accuracy
- Generation ()
- 50 t/s on Apple M2 Pro 32GB full GPU
How to run this file
From mradermacher/Sage-A1.5-i1-GGUF, file Sage-A1.5.i1-IQ3_M.gguf.
(downloads it, then starts a chat; type /bye to quit):
ollama run hf.co/mradermacher/Sage-A1.5-i1-GGUF:Sage-A1.5.i1-IQ3_M.gguf: open the Discover tab, search mradermacher/Sage-A1.5-i1-GGUF, pick the i1-IQ3_M download, then load it from the Chat tab.
- File size
- 13.2 GB
- no KLD: reference too large
- Accuracy (500 items)
- 86.8% (83.6–89.5)
- 82.0%
- Accuracy
- Generation ()
- 49 t/s on Apple M2 Pro 32GB full GPU
How to run this file
From mradermacher/Sage-A1.5-i1-GGUF, file Sage-A1.5.i1-Q2_K.gguf.
(downloads it, then starts a chat; type /bye to quit):
ollama run hf.co/mradermacher/Sage-A1.5-i1-GGUF:Sage-A1.5.i1-Q2_K.gguf: open the Discover tab, search mradermacher/Sage-A1.5-i1-GGUF, pick the i1-Q2_K download, then load it from the Chat tab.
- File size
- 15.8 GB
- no KLD: reference too large
- Accuracy (500 items)
- 88.0% (84.9–90.6)
- 84.0%
- Accuracy
- Generation ()
- 50 t/s on Apple M2 Pro 32GB full GPU
How to run this file
From mradermacher/Sage-A1.5-i1-GGUF, file Sage-A1.5.i1-IQ3_M.gguf.
(downloads it, then starts a chat; type /bye to quit):
ollama run hf.co/mradermacher/Sage-A1.5-i1-GGUF:Sage-A1.5.i1-IQ3_M.gguf: open the Discover tab, search mradermacher/Sage-A1.5-i1-GGUF, pick the i1-IQ3_M download, then load it from the Chat tab.
Other budgets: 8GB12GB24GB · all 4 Sage-A1.5 quants
How we picked
We estimate VRAM need as file size + f16 KV cache for 8192 tokens + about 0.6GB of compute buffers, and keep 0.5GB free for your desktop. Among files that fit, we rank by mean KL divergence against the BF16 model's logits, then by tool-call score. If you need a longer context, step down one quant level or quantize the KV cache (-ctk q8_0 -ctv q8_0). Methodology.
GPU too small? Rent one.
Run a 32B model at Q6/Q8, or a 70B, on a cloud GPU for an hour or two instead of buying hardware.
Cloud GPU pods with 24-80GB cards (RTX 4090 / 5090 / A100 / H100) for big quants or BF16 reference runs.
Rent on RunPodGPU marketplace for renting a 24-48GB card for an evening of testing.
Rent on Vast.aiReferral links: QuantDiff may earn credit if you sign up. Referral partners have no say in scores or rankings.