Guide · 2026-06-29

What Actually Fits in 12GB of VRAM: Measured Numbers from a 5070 Ti Laptop

By Mohamed Meguedmi · measured on an RTX 5070 Ti

Every “best local LLM” list has the same problem: the numbers come from an A100, an H100, or a 24GB desktop 4090 — then get presented as if they apply to the hardware most people actually own. They don’t. The moment your model plus its KV cache spills past your VRAM, the runtime starts streaming weights from system RAM, and behaviour changes. The interesting question on a 12GB card isn’t “what’s the best model” — it’s “what actually fits, and what happens when it doesn’t.”

So I measured it. Everything below was run on a single consumer box: an RTX 5070 Ti Laptop GPU, 12GB GDDR7 (Blackwell, sm_120), the kind of GPU a solo dev actually has. No cloud, no datacenter card.

Why 12GB is the real constraint

VRAM is consumed by three things, not one:

• Model weights — roughly params × bytes_per_param. A 7–8B model at ~4-bit is ~5GB; the same model at FP16 is ~16GB and won’t fit at all.
• KV cache — grows linearly with context length and batch size. The part everyone forgets: at 8K context a 7–8B model can need a couple of GB of KV cache on its own.
• Runtime overhead — CUDA context, allocator fragmentation, the compositor if your display shares the GPU.

So the usable budget on a 12GB card is well under 12GB once context is loaded. That’s the number to plan against.

Methodology

Honest and reproducible — the numbers are only worth anything if you can redo them:

• Hardware: RTX 5070 Ti Laptop GPU, 12GB.
• Runtime: Ollama (llama.cpp backend), default quantization per model tag (4-bit class; QAT tags are quantization-aware-trained 4-bit).
• What “tok/s” means here: steady-state generation throughput, the eval rate from ollama run --verbose on a single ~1.5–5K-token completion. Prompt-eval (prefill) is faster and not reported.
• VRAM footprint: the resident size from ollama ps (weights + KV cache at the listed context) and its GPU/CPU split.
• Context: each model’s default as loaded by Ollama (shown per row — they differ).

Single-box snapshot, batch size 1. Your mileage varies with quant, context and runtime. These are my measured numbers on this GPU, not a universal leaderboard.

The table

First, the dense ladder is exactly what you’d expect — until it isn’t. Among the dense models that fit entirely on the GPU, throughput falls as parameters rise: 8B at ~61 tok/s, 12B at ~48, 14B at ~32. The 14B (hermes4:14b) sits at 9.4GB with 8K of context — it fits comfortably, and ~32 tok/s is genuinely usable. On a 12GB card the dense 14B class is the practical ceiling: it lives on the GPU with headroom for context, and nothing larger does.

Second — the most interesting result — bigger isn’t always slower. The 26B model (gemma...:26b-a4b-it-qat) needs ~24GB, so it overflows 12GB and Ollama splits it 60% CPU / 40% GPU. By the naive “spill past VRAM = falls off a cliff” rule it should crawl. It runs at 43.3 tok/s — faster than the dense 14B that fits entirely on the GPU. Why? It’s a sparse Mixture-of-Experts: the a4b tag means only ~4B of its 26B parameters are active per token. Even with most weights parked in system RAM, only a small active slice moves per token, so the PCIe penalty is far smaller than for a dense 26B.

That’s the lesson datacenter benchmarks miss: on a VRAM-constrained box, architecture matters as much as size. A sparse MoE that “doesn’t fit” can outrun a dense model that does. The cliff is real for dense models — but MoE quietly walks around it.

So what should you run on 12GB?

• Want the fastest interactive model? A well-tuned 8B (~60 tok/s) with room for long context.
• Want the best quality that still fully fits? The dense 14B class (~32 tok/s, ~9–10GB) is the ceiling.
• Curious about punching above your VRAM? A sparse MoE is the one case where “doesn’t fit” can still be fast — measure it before you dismiss it.
• Pushing a dense 24B+ onto 12GB? That’s the real cliff. Don’t.

When local actually wins

Not always, and pretending otherwise is how people waste money. Local wins when your workload is high-volume, privacy-sensitive, or latency-tolerant; the cloud wins when you need frontier quality occasionally. The breakeven is arithmetic, not ideology. I keep a cloud-vs-local cost calculator that runs that breakeven against current API prices, because the intuition that “local is free” is usually wrong at low volume and right at high volume.

Takeaways

• Plan against your usable VRAM (well under 12GB once context loads), not the sticker number.
• The dense 14B class is the comfortable ceiling on 12GB; an 8B if you want speed and long context.
• Architecture beats size when VRAM is tight: a sparse MoE that overflows can out-run a dense model that fits.
• The cliff is real — for dense models past your VRAM. Measure your specific model and context before trusting any list, including this one.

---

Mohamed Meguedmi runs bestllmfor.com, an independent comparator of locally-runnable LLMs with benchmarks measured on his own RTX 5070 Ti workstation (data licensed CC BY 4.0). See the local LLM leaderboard.