Pick the Right Quantization for Your VRAM — Smart Recommender

The 30-second recommender

Skip the theory if you just want a number. Find your GPU's effective VRAM in the table below, then read across to the largest model and quantization that fits with an 8k context window. Numbers assume llama.cpp or Ollama defaults, Q4_K_M KV cache, and no other applications competing for memory.

The math behind the recommendation

Quantization compresses each weight from FP16 (16 bits) down to anywhere between 2 and 8 bits. The savings are deterministic; the quality loss is not. The base VRAM formula is:

VRAM_weights = Parameters × Bits_per_weight / 8

A 7B model at Q4_K_M uses roughly 4.5 bits/weight on average (the "K_M" variants keep some tensors at higher precision), giving 7,000,000,000 × 4.5 / 8 = ~3.94 GB. Add the KV cache — which scales with context length and hidden dimension — plus CUDA kernels and you land near 5.5 GB at 8k context.

The often-cited × 1.2 multiplier is a rule of thumb for 8k context. For longer contexts, the KV cache dominates and the multiplier climbs fast. Our cost calculator uses the exact per-architecture KV formulas published in the KV cache compression survey (Yuan et al., 2024).

Bits-per-weight reference

These figures are weights only. Add 15-25% for an 8k context window, 40-60% for 32k, and double the cache for 128k unless you also quantize the KV cache itself (--cache-type-k q8_0 --cache-type-v q8_0 in llama.cpp halves it with no measurable quality loss).

Quality loss: what the numbers actually say

The community has repeatedly tested how much each quantization step costs in benchmark accuracy. The pattern is consistent across Llama 3.x, Qwen3, and Mistral families.

Two observations the editorial team keeps returning to: the cliff between Q4_K_M and Q3_K_M is real and dramatic, and the gain from Q5 to Q6 is almost always invisible in production. Spend the VRAM on a bigger model instead.

A 14B at Q4_K_M beats an 8B at Q8_0 on every reasoning and coding benchmark we tracked in Q1 2026. When in doubt, scale parameters, not bits.

GGUF vs GPTQ vs AWQ: pick the right format, not just the right bit depth

Bit depth is one axis. The packing format is another, and it changes which hardware path you take.

GGUF (llama.cpp / Ollama / LM Studio)

The default for single-user inference, CPU offload, and Apple Silicon. K-quants (Q4_K_M etc.) use a mixed-precision scheme that keeps attention layers at higher precision. GGUF is what ollama.com ships and what most community quants on HuggingFace distribute.

GPTQ

4-bit only (and rarely 3-bit or 8-bit). Designed for GPU-only inference with vLLM, TGI, or AutoGPTQ. Slightly faster than GGUF on Ampere but loses ground on Ada and Blackwell where AWQ kernels are better tuned.

AWQ (Activation-aware Weight Quantization)

4-bit, GPU-only. Currently the fastest option for batched inference on RTX 4090, RTX 5090, H100, H200, and Blackwell datacenter parts. The original AWQ paper showed a 1.7-2.4× throughput edge over GPTQ on A100; on H100 the gap widened further in 2025 benchmarks.

Special cases that break the defaults

Mixture-of-Experts models

DeepSeek V3, Qwen3-MoE, and Mixtral activate only a fraction of parameters per token. The VRAM formula still uses the full parameter count for storage, but speed scales with active parameters. Quantize MoEs at Q4_K_M; lower quants damage routing decisions disproportionately. Q3 on Mixtral 8x7B costs roughly 7 percentage points on GSM8K versus Q4 — far worse than the dense-model trend.

Coding models

Code is brittle. Qwen3-Coder, DeepSeek-Coder V3, and Codestral show a sharper HumanEval drop at Q4 than chat models do. The editorial team's recommendation for paid production code agents: Q5_K_M minimum, Q6_K preferred. For local-only IDE assistants, Q4_K_M remains fine.

Long-context workloads

At 64k+ context, the KV cache is bigger than the weights. Quantize the cache before quantizing the model further. With --cache-type-k q4_0 --cache-type-v q4_0, a 32B model at Q5_K_M with 128k context fits in 32 GB — impossible with FP16 KV.

Step-by-step: download the right file

1. Measure your VRAM honestly. Use nvidia-smi or Apple's Activity Monitor. Subtract 1-2 GB for the OS and browser. That number is your budget.
2. Pick a model size from the recommender table above that leaves at least 15% headroom.
3. Choose the quantization — start at Q4_K_M, climb to Q5_K_M only if you have spare VRAM.
4. Download from a trusted quantizer. bartowski, unsloth, and the official model authors on HuggingFace are reliable. Avoid random reuploads.
5. Verify with ollama run <model> or llama-cli -m model.gguf -p "Hello". Watch nvidia-smi during a 500-token generation to confirm you're not spilling to system RAM.
6. Tune the KV cache with --cache-type-k q8_0 --cache-type-v q8_0 if you need longer contexts.

The BestLLMfor recommender API

Every number in this article is queryable through the BestLLMfor public API, released under CC BY 4.0. The endpoint /v1/quantization/recommend?vram=24&context=8192&use_case=coding returns the same JSON we use to power our cost calculator. The companion MCP server open-source MCP server lets Claude Desktop, Cursor, and other MCP clients call the recommender directly during a conversation.

Final verdict

The short answer remains unchanged from 2024: Q4_K_M is the right default. The long answer is that the right default for your workload depends on whether you serve one user or twenty, whether your context is 4k or 128k, and whether your tasks are chat, code, or agentic. Use the editorial team's recommender table as the starting point, then climb to Q5_K_M or Q6_K only when the headroom is free.