QuantDiff

Best Flint quant for your GPU

3 quantized files of RecursiveSI/flint (39.79B): accuracy, tool calls and speed (no KLD: reference too large).

RecursiveSI/flint (Apache 2.0). Released Oct 21, 2026. Recursive's mixture-of-experts coding-agent model: 39.8B total, about 2.5B active parameters per token; 49 layers (14 full attention, 35 sliding window), 256 experts with 6 selected plus a shared expert. Needs a newer llama.cpp than our other runs: tested with llama.cpp build b11869. Tool calls and accuracy tested with thinking disabled. GGUF quants from unsloth (UD dynamic); quants above about 19 GB were not tested on our 32 GB Mac.

Accuracy and speed only, no KLD: reference too large. The BF16 reference (about 80 GB) does not fit in our 32 GB test Mac (minipro), so there is no KLD for this model; we measured accuracy, tool calls and speed only. We never estimate KLD; picks for this model are ranked by measured accuracy (GSM8K + MMLU + ARC-C, 500 items, 95% CI) with tool calls as tie-break.

Charts

Every dot and bar is a measured file (GGUF). Hover or tap one for its numbers; each chart has a table under "Show the numbers". Download a PNG to share.

Flint: accuracy vs file size

If a dot's bar reaches the shaded band, that file scores the same as the best one within the margin of error on our accuracy questions (GSM8K + MMLU + ARC-C); the smallest such file here is UD-Q2_K_XL at 12.63 GB.

Flint: accuracy vs file sizeScatter of 3 GGUF files of Flint: file size in GB across, accuracy in percent up, with 95% interval bars. Shaded band: the best file's 95% interval (74.4 to 81.6%). Data table follows.60657075808581012141618File size (GB)Accuracy (% of 500 questions)UD-IQ1_M by unsloth · 10.57 GB · accuracy 66.8% (62.6–70.8) · clearly below the best · our 12 GB pickUD-Q2_K_XL by unsloth · 12.63 GB · accuracy 75.8% (71.9–79.3) · tied with the best · our 16 GB pickUD-Q3_K_XL by unsloth · 17.22 GB · accuracy 78.2% (74.4–81.6) · tied with the best · our 24 GB pickUD-IQ1_MUD-Q2_K_XLUD-Q3_K_XL
  • unsloth
  • best file's 95% range
  • clearly below the best
  • our pick for 8 / 12 / 16 GB GPUs

Up: in %, bars = · Across: file size in GB (GiB) · = overlaps the band

Show the numbers (table)
QuantGBAccuracy %95% intervalCorrectTied with bestPick
UD-IQ1_M10.5766.862.6–70.8334/500no (below)12 GB pick
UD-Q2_K_XL12.6375.871.9–79.3379/500yes16 GB pick
UD-Q3_K_XL17.2278.274.4–81.6391/500yes24 GB pick

Flint: tool calls vs file size

If a dot's bar reaches the shaded band, that file scores the same as the best one within the margin of error on our tool-call tests; the smallest such file here is UD-IQ1_M at 10.57 GB.

Flint: tool calls vs file sizeScatter of 3 GGUF files of Flint: file size in GB across, tool calls in percent up, with 95% interval bars. Shaded band: the best file's 95% interval (77.2 to 87.6%). Data table follows.65707580859081012141618File size (GB)Tool calls (% of 200 cases)UD-IQ1_M by unsloth · 10.57 GB · tool calls 82.5% (76.6–87.1) · tied with the best · our 12 GB pickUD-Q2_K_XL by unsloth · 12.63 GB · tool calls 83.0% (77.2–87.6) · tied with the best · our 16 GB pickUD-Q3_K_XL by unsloth · 17.22 GB · tool calls 77.0% (70.7–82.3) · tied with the best · our 24 GB pickUD-IQ1_MUD-Q2_K_XLUD-Q3_K_XL
  • unsloth
  • best file's 95% range
  • clearly below the best
  • our pick for 8 / 12 / 16 GB GPUs

Up: in %, bars = · Across: file size in GB (GiB) · = overlaps the band

Show the numbers (table)
QuantGBTool calls %95% intervalCorrectTied with bestPick
UD-IQ1_M10.5782.576.6–87.1165/200yes12 GB pick
UD-Q2_K_XL12.6383.077.2–87.6166/200yes16 GB pick
UD-Q3_K_XL17.2277.070.7–82.3154/200yes24 GB pick

Flint: diff from the original (KLD)

No KLD for this model: The BF16 reference (about 80 GB) does not fit in our 32 GB test Mac (minipro), so there is no KLD for this model; we measured accuracy, tool calls and speed only.

Flint: generation speed by quant and machine

Longer bar = faster replies. Smaller files are usually faster; the fastest measured here is UD-IQ1_M (unsloth) at 56 tok/s on Apple M2 Pro 32GB (llama.cpp). Compare bars of the same colour only: each colour is a different machine.

Flint: generation speed by quant and machineHorizontal bars of measured generation speed in tokens per second for 3 files on 1 machine setup(s): Apple M2 Pro 32GB (llama.cpp). Longer = faster. Data table follows.020406080Generation speed (tokens per second), higher = fasterUD-Q3_K_XLunsloth · 17.2 GBUD-Q3_K_XL by unsloth · 17.22 GB · 49.9 tok/s on Apple M2 Pro 32GB (llama.cpp)50UD-Q2_K_XLunsloth · 12.6 GBUD-Q2_K_XL by unsloth · 12.63 GB · 54.5 tok/s on Apple M2 Pro 32GB (llama.cpp)54UD-IQ1_Munsloth · 10.6 GBUD-IQ1_M by unsloth · 10.57 GB · 55.6 tok/s on Apple M2 Pro 32GB (llama.cpp)56
  • Apple M2 Pro 32GB (llama.cpp)

Across: (tokens per second)

Show the numbers (table)
FilePublisherGBMachinetok/sPlacement
UD-Q3_K_XLunsloth17.22Apple M2 Pro 32GB (llama.cpp)49.9full GPU
UD-Q2_K_XLunsloth12.63Apple M2 Pro 32GB (llama.cpp)54.5full GPU
UD-IQ1_Munsloth10.57Apple M2 Pro 32GB (llama.cpp)55.6full GPU

Not sure which fits? Use the quant picker →

Recommendation by VRAM

"Best" = highest measured accuracy (500 items pooled) among files that fit fully on the GPU (file + 8192-token KV cache), tool-call score as tie-breaker. No KLD for this model (reference too large).

8GB GPU

Best accuracy that fits

Nothing we've tested fits fully in 8GB at 8192 context. Try a smaller model, run it GPU+RAM split, or rent a bigger GPU.

No GPU big enough? Rent one on RunPod or Vast.ai.

Referral links: QuantDiff may earn credit if you sign up. Referral partners have no say in scores or rankings.

12GB GPU

Best accuracy that fits
UD-IQ1_M by unsloth
File size
10.57 GB
no KLD: reference too large
Accuracy (500 items)
66.8% (62.6–70.8)
82.5%
Accuracy
▼ below best on ARC-C
Generation ()
56 t/s on Apple M2 Pro 32GB full GPU
flint-UD-IQ1_M.gguf
How to run this file

From unsloth/flint-GGUF, file flint-UD-IQ1_M.gguf.

(downloads it, then starts a chat; type /bye to quit):

ollama run hf.co/unsloth/flint-GGUF:flint-UD-IQ1_M.gguf

: open the Discover tab, search unsloth/flint-GGUF, pick the UD-IQ1_M download, then load it from the Chat tab.

Step-by-step guide

16GB GPU

Best accuracy that fits
UD-Q2_K_XL by unsloth
File size
12.63 GB
no KLD: reference too large
Accuracy (500 items)
75.8% (71.9–79.3)
83.0%
Accuracy
Generation ()
54 t/s on Apple M2 Pro 32GB full GPU
flint-UD-Q2_K_XL.gguf
How to run this file

From unsloth/flint-GGUF, file flint-UD-Q2_K_XL.gguf.

(downloads it, then starts a chat; type /bye to quit):

ollama run hf.co/unsloth/flint-GGUF:flint-UD-Q2_K_XL.gguf

: open the Discover tab, search unsloth/flint-GGUF, pick the UD-Q2_K_XL download, then load it from the Chat tab.

Step-by-step guide

Fastest that fits: UD-IQ1_M (unsloth, 56 t/s, acc 66.8%, no KLD)

All Flint quants

QuantPublisherSizeFits meanKLD p99PP Gen Run
UD-IQ1_M unsloth imatrix 10.57 GB 12GB+ n/a n/a — 82.5% 84.7 ≈
78–90		67.0
60–73		48.7 ▼ below
41–57		 613 56 Apple M2 Pro 32GB settings & logs
UD-Q2_K_XL unsloth imatrix 12.63 GB 16GB+ n/a n/a — 83.0% 84.7 ≈
78–90		65.5 ≈
59–72		80.7 ≈
74–86		 611 54 Apple M2 Pro 32GB settings & logs
UD-Q3_K_XL unsloth imatrix 17.22 GB 24GB+ n/a n/a — 77.0% 89.3
83–93		66.5 ≈
60–73		82.7
76–88		 630 50 Apple M2 Pro 32GB settings & logs

Accuracy cells: % correct, with the 95% confidence interval (Wilson) underneath. ≈ = statistically tied with the best quant of the same model (intervals overlap). ▼ below = clearly worse than that model's best quant (intervals do not overlap). Ties are not ranked: sorting by an accuracy column puts tied quants together and orders them by KLD. How accuracy is measured.

GB = GiB (1024³ bytes). Measured on: tool calls: minipro (Apple M2 Pro 32GB); speed: minipro (Apple M2 Pro 32GB); accuracy: minipro (Metal, Apple M2 Pro (Mac mini), 32 GB unified memory). On a different GPU, expect generation speed to scale roughly with memory bandwidth. Task accuracy (GSM8K / MMLU / ARC-C, % correct on fixed subsets of 150 / 200 / 150 items, frozen suite quantdiff-lmeval-v1) was measured with the same GGUF files and llama.cpp build on Ghost (CUDA, NVIDIA GeForce RTX 5070 12 GB (64 GB RAM)), minipro (Metal, Apple M2 Pro (Mac mini), 12-core CPU, 19-core GPU, 32 GB unified memory), minipro (Metal, Apple M2 Pro (Mac mini), 32 GB unified memory). Accuracy depends on the weights, not on the GPU, but small numeric differences between backends can flip individual answers. With these sample sizes, differences under about 5 points are within noise (± shown on run pages). Methodology.

GPU too small? Rent one.

Run a 32B model at Q6/Q8, or a 70B, on a cloud GPU for an hour or two instead of buying hardware.

RunPod

Cloud GPU pods with 24-80GB cards (RTX 4090 / 5090 / A100 / H100) for big quants or BF16 reference runs.

Rent on RunPod
Vast.ai

GPU marketplace for renting a 24-48GB card for an evening of testing.

Rent on Vast.ai
Your GPU cloud here

Sponsor slot. Sponsors never influence scores or rankings.

Sponsor QuantDiff

Referral links: QuantDiff may earn credit if you sign up. Referral partners have no say in scores or rankings.