QuantDiff

Best Qwen3.8 27B quant for your GPU

4 quantized files of Qwen/Qwen3.8-27B (27.32B): accuracy, tool calls and speed (no KLD: reference too large).

Qwen/Qwen3.8-27B (Apache-2.0). Dense 27B with the Qwen3.5 hybrid design: 64 layers, 16 full attention + 48 linear attention, 4 KV heads x 256, so the KV cache is small. Text-only GGUF quants from unsloth (vision projector not included). Tool calls and accuracy tested with thinking disabled.

Accuracy and speed only, no KLD: reference too large. The BF16 reference (about 55 GB) does not fit in our 32 GB test Mac (minipro), so there is no KLD for this model; we measured accuracy, tool calls and speed only. We never estimate KLD; picks for this model are ranked by measured accuracy (GSM8K + MMLU + ARC-C, 500 items, 95% CI) with tool calls as tie-break.

Charts

Every dot and bar is a measured file (GGUF). Hover or tap one for its numbers; each chart has a table under "Show the numbers". Download a PNG to share.

Qwen3.8 27B: accuracy vs file size

If a dot's bar reaches the shaded band, that file scores the same as the best one within the margin of error on our accuracy questions (GSM8K + MMLU + ARC-C); the smallest such file here is UD-Q2_K_XL at 9.15 GB.

Qwen3.8 27B: accuracy vs file sizeScatter of 4 GGUF files of Qwen3.8 27B: file size in GB across, accuracy in percent up, with 95% interval bars. Shaded band: the best file's 95% interval (89.1 to 93.9%). Data table follows.8688909294967.510.012.515.017.520.0File size (GB)Accuracy (% of 500 questions)UD-Q5_K_M by unsloth · 18.41 GB · accuracy 91.8% (89.1–93.9) · tied with the bestUD-Q2_K_XL by unsloth · 9.15 GB · accuracy 90.4% (87.5–92.7) · tied with the best · our 12 GB pickUD-Q3_K_XL by unsloth · 12.24 GB · accuracy 91.4% (88.6–93.6) · tied with the best · our 16 GB pickUD-Q4_K_M by unsloth · 15.33 GB · accuracy 91.8% (89.1–93.9) · tied with the best · our 24 GB pickUD-Q2_K_XLUD-Q3_K_XLUD-Q4_K_MUD-Q5_K_M
  • unsloth
  • best file's 95% range
  • clearly below the best
  • our pick for 8 / 12 / 16 GB GPUs

Up: in %, bars = · Across: file size in GB (GiB) · = overlaps the band

Show the numbers (table)
QuantGBAccuracy %95% intervalCorrectTied with bestPick
UD-Q2_K_XL9.1590.487.5–92.7452/500yes12 GB pick
UD-Q3_K_XL12.2491.488.6–93.6457/500yes16 GB pick
UD-Q4_K_M15.3391.889.1–93.9459/500yes24 GB pick
UD-Q5_K_M18.4191.889.1–93.9459/500yes

Qwen3.8 27B: tool calls vs file size

If a dot's bar reaches the shaded band, that file scores the same as the best one within the margin of error on our tool-call tests; the smallest such file here is UD-Q2_K_XL at 9.15 GB.

Qwen3.8 27B: tool calls vs file sizeScatter of 4 GGUF files of Qwen3.8 27B: file size in GB across, tool calls in percent up, with 95% interval bars. Shaded band: the best file's 95% interval (83.3 to 92.2%). Data table follows.808590957.510.012.515.017.520.0File size (GB)Tool calls (% of 200 cases)UD-Q5_K_M by unsloth · 18.41 GB · tool calls 86.5% (81.1–90.6) · tied with the bestUD-Q2_K_XL by unsloth · 9.15 GB · tool calls 87.5% (82.2–91.4) · tied with the best · our 12 GB pickUD-Q3_K_XL by unsloth · 12.24 GB · tool calls 87.5% (82.2–91.4) · tied with the best · our 16 GB pickUD-Q4_K_M by unsloth · 15.33 GB · tool calls 88.5% (83.3–92.2) · tied with the best · our 24 GB pickUD-Q2_K_XLUD-Q3_K_XLUD-Q4_K_MUD-Q5_K_M
  • unsloth
  • best file's 95% range
  • clearly below the best
  • our pick for 8 / 12 / 16 GB GPUs

Up: in %, bars = · Across: file size in GB (GiB) · = overlaps the band

Show the numbers (table)
QuantGBTool calls %95% intervalCorrectTied with bestPick
UD-Q2_K_XL9.1587.582.2–91.4175/200yes12 GB pick
UD-Q3_K_XL12.2487.582.2–91.4175/200yes16 GB pick
UD-Q4_K_M15.3388.583.3–92.2177/200yes24 GB pick
UD-Q5_K_M18.4186.581.1–90.6173/200yes

Qwen3.8 27B: diff from the original (KLD)

No KLD for this model: The BF16 reference (about 55 GB) does not fit in our 32 GB test Mac (minipro), so there is no KLD for this model; we measured accuracy, tool calls and speed only.

Qwen3.8 27B: generation speed by quant and machine

Longer bar = faster replies. Smaller files are usually faster; the fastest measured here is UD-Q2_K_XL (unsloth) at 48 tok/s on RTX 5070 12GB (llama.cpp). Compare bars of the same colour only: each colour is a different machine.

Qwen3.8 27B: generation speed by quant and machineHorizontal bars of measured generation speed in tokens per second for 4 files on 2 machine setup(s): RTX 5070 12GB (llama.cpp), Apple M2 Pro 32GB (llama.cpp). Longer = faster. Data table follows.020406080Generation speed (tokens per second), higher = fasterUD-Q5_K_Munsloth · 18.4 GBUD-Q5_K_M by unsloth · 18.41 GB · 7.6 tok/s on Apple M2 Pro 32GB (llama.cpp)8UD-Q4_K_Munsloth · 15.3 GBUD-Q4_K_M by unsloth · 15.33 GB · 8.8 tok/s on Apple M2 Pro 32GB (llama.cpp)9UD-Q3_K_XLunsloth · 12.2 GBUD-Q3_K_XL by unsloth · 12.24 GB · 9.4 tok/s on Apple M2 Pro 32GB (llama.cpp)9UD-Q2_K_XLunsloth · 9.2 GBUD-Q2_K_XL by unsloth · 9.15 GB · 48.1 tok/s on RTX 5070 12GB (llama.cpp)48UD-Q2_K_XL by unsloth · 9.15 GB · 9.7 tok/s on Apple M2 Pro 32GB (llama.cpp)10
  • RTX 5070 12GB (llama.cpp)
  • Apple M2 Pro 32GB (llama.cpp)

Across: (tokens per second)

Show the numbers (table)
FilePublisherGBMachinetok/sPlacement
UD-Q5_K_Munsloth18.41Apple M2 Pro 32GB (llama.cpp)7.6full GPU
UD-Q4_K_Munsloth15.33Apple M2 Pro 32GB (llama.cpp)8.8full GPU
UD-Q3_K_XLunsloth12.24Apple M2 Pro 32GB (llama.cpp)9.4full GPU
UD-Q2_K_XLunsloth9.15RTX 5070 12GB (llama.cpp)48.1full GPU
UD-Q2_K_XLunsloth9.15Apple M2 Pro 32GB (llama.cpp)9.7full GPU

Not sure which fits? Use the quant picker →

Recommendation by VRAM

"Best" = highest measured accuracy (500 items pooled) among files that fit fully on the GPU (file + 8192-token KV cache), tool-call score as tie-breaker. No KLD for this model (reference too large).

8GB GPU

Best accuracy that fits

Nothing we've tested fits fully in 8GB at 8192 context. Try a smaller model, run it GPU+RAM split, or rent a bigger GPU.

No GPU big enough? Rent one on RunPod or Vast.ai.

Referral links: QuantDiff may earn credit if you sign up. Referral partners have no say in scores or rankings.

12GB GPU

Best accuracy that fits
UD-Q2_K_XL by unsloth
File size
9.15 GB
no KLD: reference too large
Accuracy (500 items)
90.4% (87.5–92.7)
87.5%
Accuracy
Generation ()
10 t/s on Apple M2 Pro 32GB full GPU
Qwen3.8-27B-UD-Q2_K_XL.gguf
How to run this file

From unsloth/Qwen3.8-27B-GGUF, file Qwen3.8-27B-UD-Q2_K_XL.gguf.

(downloads it, then starts a chat; type /bye to quit):

ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:Qwen3.8-27B-UD-Q2_K_XL.gguf

: open the Discover tab, search unsloth/Qwen3.8-27B-GGUF, pick the UD-Q2_K_XL download, then load it from the Chat tab.

Step-by-step guide

16GB GPU

Best accuracy that fits
UD-Q3_K_XL by unsloth
File size
12.24 GB
no KLD: reference too large
Accuracy (500 items)
91.4% (88.6–93.6)
87.5%
Accuracy
Generation ()
9 t/s on Apple M2 Pro 32GB full GPU
Qwen3.8-27B-UD-Q3_K_XL.gguf
How to run this file

From unsloth/Qwen3.8-27B-GGUF, file Qwen3.8-27B-UD-Q3_K_XL.gguf.

(downloads it, then starts a chat; type /bye to quit):

ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:Qwen3.8-27B-UD-Q3_K_XL.gguf

: open the Discover tab, search unsloth/Qwen3.8-27B-GGUF, pick the UD-Q3_K_XL download, then load it from the Chat tab.

Step-by-step guide

Fastest that fits: UD-Q2_K_XL (unsloth, 10 t/s, acc 90.4%, no KLD)

All Qwen3.8 27B quants

QuantPublisherSizeFits meanKLD p99PP Gen Run
UD-Q2_K_XL unsloth imatrix 9.15 GB 12GB+ n/a n/a — 87.5% 96.7
92–99		81.0 ≈
75–86		96.7
92–99		 90 10 Apple M2 Pro 32GB
48 on RTX 5070 12GB		 settings & logs
UD-Q3_K_XL unsloth imatrix 12.24 GB 16GB+ n/a n/a — 87.5% 96.0 ≈
92–98		84.5 ≈
79–89		96.0 ≈
92–98		 90 9 Apple M2 Pro 32GB settings & logs
UD-Q4_K_M unsloth imatrix 15.33 GB 24GB+ n/a n/a — 88.5% 96.7
92–99		85.0 ≈
79–89		96.0 ≈
92–98		 89 9 Apple M2 Pro 32GB settings & logs
UD-Q5_K_M unsloth imatrix 18.41 GB 24GB+ n/a n/a — 86.5% 96.0 ≈
92–98		85.5
80–90		96.0 ≈
92–98		 85 8 Apple M2 Pro 32GB settings & logs

Accuracy cells: % correct, with the 95% confidence interval (Wilson) underneath. ≈ = statistically tied with the best quant of the same model (intervals overlap). ▼ below = clearly worse than that model's best quant (intervals do not overlap). Ties are not ranked: sorting by an accuracy column puts tied quants together and orders them by KLD. How accuracy is measured.

GB = GiB (1024³ bytes). Measured on: tool calls: minipro (Apple M2 Pro 32GB); speed: RTX 5070 12GB, minipro (Apple M2 Pro 32GB); accuracy: minipro (Metal, Apple M2 Pro (Mac mini), 32 GB unified memory). On a different GPU, expect generation speed to scale roughly with memory bandwidth. Task accuracy (GSM8K / MMLU / ARC-C, % correct on fixed subsets of 150 / 200 / 150 items, frozen suite quantdiff-lmeval-v1) was measured with the same GGUF files and llama.cpp build on Ghost (CUDA, NVIDIA GeForce RTX 5070 12 GB (64 GB RAM)), minipro (Metal, Apple M2 Pro (Mac mini), 12-core CPU, 19-core GPU, 32 GB unified memory), minipro (Metal, Apple M2 Pro (Mac mini), 32 GB unified memory). Accuracy depends on the weights, not on the GPU, but small numeric differences between backends can flip individual answers. With these sample sizes, differences under about 5 points are within noise (± shown on run pages). Methodology.

GPU too small? Rent one.

Run a 32B model at Q6/Q8, or a 70B, on a cloud GPU for an hour or two instead of buying hardware.

RunPod

Cloud GPU pods with 24-80GB cards (RTX 4090 / 5090 / A100 / H100) for big quants or BF16 reference runs.

Rent on RunPod
Vast.ai

GPU marketplace for renting a 24-48GB card for an evening of testing.

Rent on Vast.ai
Your GPU cloud here

Sponsor slot. Sponsors never influence scores or rankings.

Sponsor QuantDiff

Referral links: QuantDiff may earn credit if you sign up. Referral partners have no say in scores or rankings.