Apple Silicon (MLX)
MLX-format quants (the format LM Studio and mlx-lm use on Macs) of the same models, tested on a Mac mini with an Apple M2 Pro (19-core GPU, 32GB). This is a separate section: these speeds are M2 Pro numbers and are never compared with the RTX 5070 GGUF speeds on the rest of the site.
- KLD: vs the BF16 MLX model of the same weights, over the full vocabulary, on the same
quantdiff-kld-v1text and windows as the GGUF KLD (100 × 512 tokens, second half of each window scored), computed with our script kld_mlx.py (reference log-probabilities stored as float16). Compare MLX quants with each other. GGUF KLD numbers come from llama.cpp against a different (GGUF) BF16 reference, so they are close in spirit but not the same measurement. - Accuracy: the identical frozen suite and settings as the GGUF quants (
quantdiff-lmeval-v1, lm-eval 0.4.13, temperature 0, thinking off, 4 concurrent requests), served bymlx_lm.server. 95% intervals and tie flags work as on the rest of the site, within this table. - Tool calls: the same 200 BFCL cases and scorer, via
mlx_lm.server's OpenAI-compatible tools API. - Speed:
mlx_lm.benchmark -p 512 -g 128 -n 5 -b 1(512-token prompt, 128 generated tokens, mean of 5 trials), comparable in setup to llama-bench pp512/tg128, on the M2 Pro only. - "not measured" means we don't have that number; we never fill gaps.
Qwen3.5 9B
| MLX quant | Size | KLD vs BF16 MLX | Same top token | GSM8K | MMLU | ARC-C | Tool calls | PP t/s (M2 Pro) | Gen t/s (M2 Pro) |
|---|---|---|---|---|---|---|---|---|---|
| mlx-community/Qwen3.5-9B-4bit | 5.57 GB | 0.1010 | 86.8% | 90.7 ≈ 85–94 | held (scoring issue) | held (scoring issue) | 93.5% | 254 | 37.9 |
| mlx-community/Qwen3.5-9B-5bit | 6.61 GB | 0.0329 | 92.8% | 92.0 87–95 | held (scoring issue) | held (scoring issue) | 91.0% | 247 | 31.4 |
| mlx-community/Qwen3.5-9B-6bit | 7.65 GB | 0.0096 | 96.1% | 92.0 87–95 | held (scoring issue) | held (scoring issue) | 91.5% | 245 | 26.1 |
Accuracy cells: % correct, with the 95% confidence interval (Wilson) underneath. ≈ = statistically tied with the best quant of the same model (intervals overlap). ▼ below = clearly worse than that model's best quant (intervals do not overlap). Ties are not ranked: sorting by an accuracy column puts tied quants together and orders them by KLD. How accuracy is measured.
Software: mlx 0.32.4, mlx-lm 0.32.2, lm-eval 0.4.13. Machine: Apple M2 Pro (Mac mini), 12-core CPU, 19-core GPU, 32 GB unified memory, macOS 27.0.1. Exact repos and revisions are linked; per-quant logs (lm-eval samples, tool-call transcripts, benchmark output, KLD) are at /logs/mlx/<id>/ and in /data/mlx.json.