Best Nemotron 3.5 Bolt E4B quant for your GPU

8 quantized files of nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 (7.52B), each diffed against the BF16 original.

nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 (OpenMDW 1.1; Gemma 4 terms also apply). NVIDIA fine-tune of google/gemma-4-E4B-it, released Oct 23, 2026 per its model card: 42 layers (7 global attention, the rest 512-token sliding window), the last 18 layers reuse earlier KV caches, so the KV cache is small. Text-only GGUF quants from unsloth (vision projector not included; unsloth only publishes UD dynamic quants for this model). Tool calls and accuracy tested with thinking disabled. KLD is measured on a chat-format corpus (Gemma 4 turn template) because raw wikitext is out of distribution for this model (BF16 perplexity 72.6); chat-format KLD is not comparable across models.

Charts

Every dot and bar is a measured file (GGUF). Hover or tap one for its numbers; each chart has a table under "Show the numbers". Download a PNG to share.

Not sure which fits? Use the quant picker →

Recommendation by VRAM

"Best" = smallest diff (lowest mean KLD) among files that fit fully on the GPU (file + 8192-token KV cache), with tool-call score as tie-breaker.

All Nemotron 3.5 Bolt E4B quants

Accuracy cells: % correct, with the 95% confidence interval (Wilson) underneath. ≈ = statistically tied with the best quant of the same model (intervals overlap). ▼ below = clearly worse than that model's best quant (intervals do not overlap). Ties are not ranked: sorting by an accuracy column puts tied quants together and orders them by KLD. How accuracy is measured.

GB = GiB (1024³ bytes). Measured on: KLD: minipro (Apple M2 Pro 32GB); tool calls: minipro (Apple M2 Pro 32GB); speed: minipro (Apple M2 Pro 32GB); accuracy: minipro (Metal, Apple M2 Pro (Mac mini), 32 GB unified memory). On a different GPU, expect generation speed to scale roughly with memory bandwidth. Task accuracy (GSM8K / MMLU / ARC-C, % correct on fixed subsets of 150 / 200 / 150 items, frozen suite quantdiff-lmeval-v1) was measured with the same GGUF files and llama.cpp build on Ghost (CUDA, NVIDIA GeForce RTX 5070 12 GB (64 GB RAM)), minipro (Metal, Apple M2 Pro (Mac mini), 12-core CPU, 19-core GPU, 32 GB unified memory), minipro (Metal, Apple M2 Pro (Mac mini), 32 GB unified memory). Accuracy depends on the weights, not on the GPU, but small numeric differences between backends can flip individual answers. With these sample sizes, differences under about 5 points are within noise (± shown on run pages). Methodology.