New to this? Start here

This page explains, in plain words, what QuantDiff tests and how to get one of our picks running on your own computer. It takes about five minutes to read. Words with a dotted underline open a short explanation when you tap them; all of them are on the glossary page.

What is a local model?

A local model is an AI chat model that runs on your own computer instead of on a company's server. You download it once. After that it works offline, it is free to use, and your chats stay on your machine. You need a graphics card (GPU) or an Apple Silicon Mac with enough memory, and a free app such as LM Studio or Ollama.

What is a quant?

The original model is big: it stores every number with 16 bits (BF16). That is often too large for a normal graphics card. A quant is a compressed copy that uses fewer bits per number, for example about 8, 6, 4 or even 2. Fewer bits means a smaller file that loads on more hardware and usually runs faster. Squeeze too hard, though, and the model starts making more mistakes.

Quant files have names like Q4_K_M or IQ4_XS. The number after Q is roughly the bits per number: Q8 is big and close to the original, Q4 is about half that size, Q2 is tiny. Prefixes such as UD- or i1- mean the maker used a smarter recipe. Different people (unsloth, bartowski, mradermacher and others) publish their own quants of the same model on Hugging Face, and they are not always equal. That is why we test the actual files.

QuantDiff measures each file against the original: how much its word predictions drift (KLD), whether it still answers test questions correctly (MMLU, ARC, GSM8K), whether it can still use tools (tool calls), and how fast it runs (tokens per second).

GGUF or MLX: which one do you need?

A GGUF quant is one file. An MLX quant is a folder (one Hugging Face repo per quant). If the picker gives you an MLX pick, use LM Studio or mlx-lm; Ollama is for GGUF files here. Macs share one pool of unified memory between the CPU and GPU, so a Mac with lots of memory can run bigger models than a graphics card with less memory. See Apple Silicon results.

How to read a pick

Tell the picker your graphics card (or Mac), what you'll use the model for, and how much context you want. It shows only files that fit fully in your memory, with room for the conversation (KV cache). Then it shows up to two answers:

• Top pick: the file that scored highest on the tests for your use, among files that fit.
• Smaller & faster tied option: a smaller file whose score is statistically tied with the top pick. Choose it if you want more speed or more room for long chats.

"Statistically tied" means the two scores are so close that the gap could be luck. We test a fixed sample of questions, so every score has a 95% interval: the range where the true score most likely lies. If two files' intervals overlap, we can't honestly say one is better, so we call them tied.

How to run a pick

Under every pick on this site there is a How to run this file box with the exact Hugging Face repo and file name, a copy button for the Ollama command and the LM Studio search term. Here is what to do with them.

LM Studio (Windows, Mac, Linux; easiest)

1. Download LM Studio from lmstudio.ai and install it.
2. Open the Discover tab (the magnifying glass).
3. Search for the repo name from the pick, for example unsloth/Qwen3.8-27B-GGUF.
4. Choose the download option that matches the quant in the pick, for example UD-Q2_K_XL, and click Download.
5. Go to the Chat tab, open the model loader at the top, and select the model you downloaded.
6. Type a message. That's it.

On a Mac, LM Studio can run MLX picks too: search the MLX repo name the same way. Source: LM Studio docs.

Ollama (Windows, Mac, Linux; command line)

1. Install Ollama from ollama.com/download.
2. Open a terminal (on Windows: PowerShell) and paste the Ollama command from the pick's run box. It looks like this:
   ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:Qwen3.8-27B-UD-Q2_K_XL.ggufCopy
   Ollama downloads the file straight from Hugging Face, then starts a chat.
3. Type your message. Type /bye to quit.

The part after the colon picks the exact file in the repo. If a brand-new model won't load, update Ollama first. Ollama's default context window can be smaller than our 8,192-token assumption; the Ollama FAQ explains how to change it with the OLLAMA_CONTEXT_LENGTH setting. Sources: Hugging Face: use GGUF with Ollama, Ollama quickstart, Ollama FAQ.

MLX on a Mac with mlx-lm (command line)

1. Open Terminal and install mlx-lm (needs Python):
   pip install mlx-lmCopy
2. Start a chat with the MLX repo from the pick, for example:
   mlx_lm.chat --model mlx-community/Qwen3.5-9B-4bitCopy
   The first run downloads the model.

Source: mlx-lm on GitHub.

FAQ

Is it free?

LM Studio, Ollama and mlx-lm are free to download, and so are the model files we test. Check each model's license if you use it for work.

Will it run on my computer?

Put your graphics card or Mac into the picker. It only suggests files that fit fully in your memory. If nothing fits, try a smaller model or a shorter context.

Which file should I pick if I'm not sure?

Take the picker's top pick. If it feels slow, try the smaller & faster tied option: on our tests it scored about the same.

Why not always take the biggest file?

A bigger file needs more memory and runs slower. On our tests, mid-size files often scored the same as the biggest ones. The home page and the leaderboard show the numbers.

What does "KLD pending" mean?

We haven't finished measuring that file's KLD yet. Its other scores are real; the KLD number will appear when it's done.

Is this as good as ChatGPT?

We don't compare against cloud services. We compare each quant with its own original model, so you know how much quality the smaller file gives up.

Where do the numbers come from?

From our own test runs, with settings and raw logs published for each one. See the methodology and the open data.

Words still confusing? Try the glossary.