Guide · 2026-05-16

LLM VRAM Requirements — Every Model, Every Quant (2026)

Exact VRAM numbers for every major local LLM in 2026, from Qwen3 4B on a laptop iGPU to DeepSeek-R1 671B on an 8×H200 node.

By Mohamed Meguedmi · 11 min read

How to read a VRAM number in 2026

Every “VRAM cheat sheet” you find on Reddit gets one of three things wrong: it ignores the KV cache, it assumes FP16 weights when nobody runs FP16 anymore, or it confuses VRAM load with VRAM peak. The editorial team measured every model in this article on a controlled bench (single RTX 5090, single H100 80 GB, and an M3 Ultra 192 GB for the giants), running llama.cpp build b6291, vLLM 0.8.2, and MLX 0.21 with a fixed 4K prompt and 1K generation.

The working formula:

VRAM_total = W_quant + KV + activations + runtime

W_quant     = params × (bits_per_weight / 8) × 1.08   // 8% overhead for metadata/scales
KV (fp16)   = 2 × n_layers × n_kv_heads × head_dim × ctx × 2 bytes
activations ≈ 0.3 – 0.8 GB at batch 1
runtime     ≈ 0.6 – 1.2 GB (CUDA context, allocator slack)

For a Qwen3.5 32B at Q4_K_M with 16K context: weights = 32 × 0.5 × 1.08 = 17.3 GB, KV at fp16 ≈ 6.1 GB, plus ~1 GB overhead → ~24.5 GB. That’s why an RTX 5090 (32 GB) is the right card for this model, not a 4090. The community Qwen3.5 32B GGUF card publishes near-identical numbers.

The master table — weights only, all quants

Numbers below are weights only, rounded, with the 8% format overhead baked in. Add KV cache (next section) for total VRAM.

KV cache: the table everyone forgets

Context length costs real memory. With Grouped-Query Attention (GQA) becoming standard in 2026, the KV cache is smaller than the Llama-2 era, but it still scales linearly with tokens. Quantizing the KV cache to Q8_0 (vLLM --kv-cache-dtype fp8 or llama.cpp -ctk q8_0 -ctv q8_0) cuts this in half with no measurable quality loss on long-context evals.

Two consequences. First, MLA is the single biggest reason DeepSeek-R1 671B is even runnable on a single 8×H100 node — without it the 128K KV would exceed total VRAM. Second, on a 24 GB GPU, the difference between running Qwen3.5 32B at 8K versus 32K context is roughly 4 GB — the difference between “fits with headroom” and “OOM on long prompts.”

Which quant should you actually pick?

Bartowski’s quants on HuggingFace are the de facto standard, but the matrix of K_S / K_M / K_L / IQ variants confuses everyone. Our position, after re-running MMLU-Pro and HumanEval on every model in the table:

• Q4_K_M — default. <1.5% MMLU-Pro drop vs FP16 on every model 7B and up.
• Q5_K_M — only if you have 20–30% headroom and run code or math. The Qwen3-Coder family in particular recovers ~0.8 points on HumanEval+ at Q5 vs Q4.
• Q6_K — pointless on dense models. Use it for 4B-and-under tiny models where the absolute VRAM delta is sub-gigabyte.
• IQ3_XXS / IQ2_M — emergency only. DeepSeek-R1 671B at IQ2_XXS (~190 GB) is the famous “runs on an M3 Ultra” build, but quality drops are visible on reasoning chains beyond 4K tokens.
• Q8_0 and FP16 — you are paying double VRAM for a benchmark delta inside the noise floor. Skip unless you’re doing research on quantization itself.

For full benchmark methodology see our methodology page; the per-quant scores are also queryable through the public API.

GPU tier picks — the verdict

Special cases: MoE, vision, and reasoning

Mixture-of-Experts

Llama 4 Maverick (120B, 17B active) and Qwen3-MoE 235B (22B active) are the practical MoE picks in 2026. Their VRAM footprint is dictated by total parameters — you have to load every expert — but their compute footprint scales with active parameters, so they decode 2–3× faster than a dense model of equivalent size. On an M3 Ultra, Llama 4 Maverick at Q4_K_M is the only frontier-class model that decodes above 10 t/s — a dense 120B can’t.

Reasoning models

DeepSeek-R1, Qwen3-Reasoning, and o-style derivatives spend 2K–30K thinking tokens before answering. KV-cache budget matters more than for instruct models: a 32B reasoner at 32K effective context needs the full ~24 GB of an RTX 5090, with no headroom for batching. Plan one tier up than you would for instruct.

Vision-language models

Add ~1.5 GB for the vision tower regardless of base model size. Qwen3-VL 32B Q4_K_M lands at ~26 GB total — tight but doable on a 5090.

How to install and verify in 10 minutes

1. Install Ollama: curl -fsSL https://ollama.com/install.sh | sh
2. Pull a model matching your tier from the table above, e.g. ollama pull qwen3.5:32b-q4_K_M.
3. Measure actual VRAM: nvidia-smi --query-gpu=memory.used --format=csv -l 1 in a side terminal while you run a 1K-token prompt.
4. Check KV scaling: rerun with OLLAMA_CONTEXT_LENGTH=32768 and watch VRAM grow by roughly the KV row in our table.
5. Compare to the predicted number from the formula at the top of this article. Discrepancy > 10% almost always means another process is holding VRAM — check fuser -v /dev/nvidia*.

For deeper automation, the open-source quelllm-mcp server exposes this VRAM database as an MCP tool, so Claude Desktop or any MCP client can answer “will this model fit?” without leaving your editor. The French sister site quelllm.fr mirrors the same dataset for the Francophone audience.

Editorial verdict

For 2026, the editorial position is unchanged from Q1: buy for the model, not the brand. An RTX 5090 at $2,400 is the highest tokens-per-dollar GPU for the 24–32 GB band any reasonable user will land in. Above that, Apple Silicon’s 192 GB unified memory is the only sub-$10K path to running Llama 4 Maverick locally, and the M3 Ultra delivers 60–70% of an H100’s decode throughput on MoE workloads at a fifth of the price. Below 12 GB, accept that you are running 4B–8B models and pick Qwen3 4B Q5_K_M — its 78 t/s on a 5060 outperforms every alternative for assistant-class tasks.

The team updates these numbers monthly. For the always-current dataset, the BestLLMfor public API returns JSON for every (model, quant, context, GPU) tuple, free under CC BY 4.0.