LLM VRAM calculator.
How much GPU memory does a local LLM really need? Weights + KV cache + overhead, in one honest estimate.
The rule of thumb: VRAM ≈ parameters (B) × bytes per weight + KV cache + ~20% overhead. At Q4 quantization that is roughly 0.55–0.6 GB per billion parameters; at FP16, 2 GB per billion. Longer contexts grow the KV cache well past the 20% budget.
Estimate = weights × (1 + 0.2 × context/8K). Dense-model heuristic; sparse MoE models activate fewer weights per token and can behave better than their size suggests. Always leave 1–2 GB headroom for the OS and display.
VRAM needed by model size (8K context)
| Model size | Q4_K_M | Q8_0 | FP16 | Fits on (Q4) |
|---|---|---|---|---|
| 7B | 4.9 GB | 9.0 GB | 16.8 GB | 8GB cards (RTX 3050, 4060) |
| 9B | 6.3 GB | 11.6 GB | 21.6 GB | 8–12GB cards |
| 14B | 9.7 GB | 18.0 GB | 33.6 GB | 12–16GB cards |
| 27B | 18.8 GB | 34.7 GB | 64.8 GB | 24GB cards (RTX 3090/4090) |
| 32B | 22.3 GB | 41.1 GB | 76.8 GB | 24GB cards, short context |
| 70B | 48.7 GB | 89.9 GB | 168.0 GB | 48GB+ or dual 24GB GPUs |
Figures include the 20% KV-cache/overhead budget at 8K context, rounded to 0.1 GB.
Now pick the model
Knowing the budget is half the job. The hardware configurator matches real models to your exact GPU or Mac; the local LLM leaderboard ranks every locally-runnable model; and the per-tier rankings go straight to the point: best LLM for 8GB VRAM, RTX 4090, 16GB Mac. Weighing cloud vs local? Run the numbers in the cost calculator. Our estimates are cross-checked against models benchmarked on a real RTX 5070 Ti — see what actually fits in 12GB.
Frequently asked questions
How much VRAM do I need per billion parameters?
Roughly 2 GB per billion parameters at FP16, about 1.1 GB per billion at Q8, and 0.55-0.6 GB per billion at Q4 quantization. On top of the weights, budget around 20% extra for the KV cache and runtime overhead at a standard 8K context, more for longer contexts.
Is 8GB of VRAM enough to run a local LLM?
Yes. At Q4 quantization an 8-9B model (about 5-6 GB of weights) fits fully on an 8GB GPU with room for context. See our ranking of the best LLMs for 8GB VRAM for concrete picks.
Can a 24GB GPU like the RTX 3090 or RTX 4090 run a 32B model?
Yes, at Q4 quantization a 32B dense model needs roughly 20-22 GB, which fits on a single 24GB card with a short context. For long contexts, a 27B-class model is the safer fit.
Do Macs count unified memory as VRAM?
Apple Silicon Macs share one pool of unified memory between CPU and GPU, so a 32GB Mac can load bigger models than a 16GB discrete GPU — but reserve 8-10 GB for macOS itself. Our Mac rankings pick models per RAM tier.