Best Qwen3.8 27B quant for a 8GB GPU
No tested Qwen3.8 27B quant fits fully in 8GB VRAM with a 8192-token context. You can still run it with a GPU+RAM split (much slower), pick a smaller model, or rent a larger GPU.
Nothing we've tested fits fully in 8GB at 8192 context. Try a smaller model, run it GPU+RAM split, or rent a bigger GPU.
Nothing we've tested fits fully in 8GB at 8192 context. Try a smaller model, run it GPU+RAM split, or rent a bigger GPU.
Other budgets: 12GB16GB24GB · all 4 Qwen3.8 27B quants
How we picked
We estimate VRAM need as file size + f16 KV cache for 8192 tokens + about 0.6GB of compute buffers, and keep 0.5GB free for your desktop. Among files that fit, we rank by mean KL divergence against the BF16 model's logits, then by tool-call score. If you need a longer context, step down one quant level or quantize the KV cache (-ctk q8_0 -ctv q8_0). Methodology.
GPU too small? Rent one.
Run a 32B model at Q6/Q8, or a 70B, on a cloud GPU for an hour or two instead of buying hardware.
Cloud GPU pods with 24-80GB cards (RTX 4090 / 5090 / A100 / H100) for big quants or BF16 reference runs.
Rent on RunPodGPU marketplace for renting a 24-48GB card for an evening of testing.
Rent on Vast.aiReferral links: QuantDiff may earn credit if you sign up. Referral partners have no say in scores or rankings.