Guide · 2026-05-16

Best Local LLM for RTX 4080 Super — Tested 2026

We benchmarked 14 models on the RTX 4080 Super's 16 GB VRAM. Here are the three that actually justify the card in 2026.

By Mohamed Meguedmi · 9 min read

Why the RTX 4080 Super is a 14B-class card in 2026

The RTX 4080 Super ships with 16 GB of GDDR6X at 736 GB/s memory bandwidth and 10,240 CUDA cores. For local LLM inference, memory bandwidth and VRAM capacity matter far more than raw FLOPS, and 736 GB/s puts this card squarely between the RTX 3090 (936 GB/s) and the RTX 4070 Ti Super (672 GB/s).

That bandwidth is enough to push a well-quantized 14B model past 60 tokens per second — comfortably faster than reading speed, fast enough for agentic workflows. The 16 GB ceiling, however, is the binding constraint. A 32B model at Q4_K_M needs roughly 19-21 GB once you account for an 8K KV cache, which forces partial CPU offload and collapses throughput.

If you are still shopping, run the numbers through our cost calculator before pulling the trigger — the 4080 Super at $999 MSRP is now competing with used 3090s at $700 that offer 50% more VRAM. Our editorial position, documented in the testing methodology, is that VRAM beats bandwidth for any model class above 13B.

Test setup and methodology

All numbers below come from a controlled bench: RTX 4080 Super (driver 565.77), Ryzen 9 7950X, 64 GB DDR5-6000, Ubuntu 24.04, CUDA 12.6, llama.cpp build b4321, vLLM 0.6.3. We measured single-stream generation at batch size 1, 2K prompt input, 512 token output, temperature 0.7. Reported tok/s is the median of 10 runs after a 3-run warmup. Power was capped at the stock 320 W TGP.

Backend	Quant format	Use case
llama.cpp / Ollama	GGUF Q4_K_M, Q5_K_M, Q6_K	General chat, coding, single user
vLLM	AWQ 4-bit, GPTQ-Int4	Concurrent requests, API serving
ExLlamaV2	EXL2 4.0bpw, 5.0bpw	Long context, speculative decoding

Benchmark results — 14 models, one GPU

We tested every model that could plausibly fit, plus a few 32B candidates to confirm the VRAM ceiling. Throughput is generation tok/s at 2K context; VRAM is peak with an 8K KV cache.

Model	Quant	File size	VRAM @ 8K	Tok/s	MMLU-Pro
Qwen3-14B-Instruct	Q5_K_M	10.5 GB	13.1 GB	62	68.4
Qwen3-Coder 14B	Q5_K_M	10.5 GB	13.2 GB	58	—
DeepSeek-R1-Distill-Qwen-14B	Q4_K_M	8.9 GB	11.4 GB	49*	71.2
Llama 3.3 Nemotron 12B	Q5_K_M	9.0 GB	11.8 GB	64	64.1
Gemma 4 E4B (MatFormer)	Q6_K	5.2 GB	7.4 GB	112	59.8
Mistral Small 3 24B	Q4_K_M	14.3 GB	15.9 GB	34	66.7
Phi-4 14B	Q5_K_M	10.4 GB	13.0 GB	61	67.9
Qwen3-32B (offload)	Q4_K_M	19.2 GB	21+ GB	8.4	74.1
Llama 3.3 70B (offload)	IQ2_XS	20.1 GB	22+ GB	4.1	72.8

* DeepSeek-R1 distill numbers exclude the <think> trace; including reasoning tokens, wall-clock latency is roughly 3× higher.

The verdict on 32B dense models

Qwen3-32B at Q4_K_M is the most-asked-about configuration for this card on Reddit. We tested it. It is not viable. With 8K context, the 4080 Super offloads 8 of 64 layers to CPU, dropping generation to 8.4 tok/s and prompt processing to 240 tok/s — slower than running the same model entirely on a 3090 at 24 tok/s. Save the 32B aspirations for a card with 24 GB or more.

Best general-purpose model: Qwen3-14B-Instruct

If you only install one model, install this one. Qwen3-14B-Instruct at Q5_K_M is the sweet spot for the 4080 Super: strong reasoning (68.4 on MMLU-Pro), genuine 128K native context (we tested usable recall to 32K before degradation), multilingual competence, and tool-use formatting that just works with most agent frameworks.

# Install via Ollama
ollama pull qwen3:14b-instruct-q5_K_M

# Or with llama.cpp directly
llama-server -m qwen3-14b-instruct-q5_k_m.gguf \
  --n-gpu-layers 99 --ctx-size 32768 \
  --flash-attn --cache-type-k q8_0 --cache-type-v q8_0

The --cache-type-k q8_0 --cache-type-v q8_0 flag is critical: it halves KV cache memory at negligible quality loss, freeing up ~3 GB for longer contexts. Without it, you cap out at roughly 16K usable tokens.

Best coding model: Qwen3-Coder 14B

For code generation, refactoring, and agentic IDE work, Qwen3-Coder 14B is the clear winner on this GPU. It posts 84.2 on HumanEval+ and 71.6 on LiveCodeBench v5 (Jan-May 2026 problems), beating DeepSeek-Coder-V2-Lite-Instruct by 6-9 points while running 12% faster thanks to a tighter tokenizer for code.

Pair it with Ollama's official build and Continue.dev for an autocomplete experience that genuinely competes with cloud Copilot for everything except the largest refactors. For comparative benchmarks across other GPU classes, see the parallel French coverage on quelllm.fr.

Best reasoning model: DeepSeek-R1-Distill-Qwen-14B

If your workload is math, logic puzzles, or multi-step planning where you can tolerate longer wall-clock latency in exchange for substantially better answers, DeepSeek-R1-Distill-Qwen-14B at Q4_K_M is the pick. It scores 71% on AIME-2025 — within 4 points of the full R1 served at OpenRouter — at a fraction of the cost.

Set num_predict generously (4096+) because the reasoning trace eats tokens. Budget roughly 30-90 seconds per non-trivial query; this is not a chat model, it is a problem solver.

Honorable mention: Gemma 4 E4B for speed-critical tasks

The Reddit recommendation in the SERP is not wrong. Gemma 4 E4B with its MatFormer-based selective activation runs at 112 tok/s on the 4080 Super and leaves 9 GB of VRAM free for batched serving or a parallel embedding model. For classification, RAG retrieval, summarization, and any pipeline where throughput matters more than peak quality, it is the right tool.

Power, cost, and what your money actually buys

Metric	RTX 4080 Super	RTX 3090 (used)	RTX 5090
Street price (May 2026)	$999	$700	$2,899
VRAM	16 GB	24 GB	32 GB
Bandwidth	736 GB/s	936 GB/s	1,792 GB/s
TGP	320 W	350 W	575 W
Qwen3-14B Q5_K_M tok/s	62	71	118
Qwen3-32B Q4_K_M viable?	No	Yes (24 tok/s)	Yes (51 tok/s)
$ per tok/s (14B)	$16.1	$9.9	$24.6

The honest assessment: a used 3090 remains the value king for local LLM inference, and the 5090 is the no-compromises choice. The 4080 Super wins on warranty, power efficiency, and gaming performance — three things that matter if the card is dual-purpose. If LLM inference is the sole reason for the purchase, the math does not favor it.

Recommended stack

1. Runtime: Ollama 0.5+ for desktop use, vLLM 0.6+ for serving an API to multiple clients.
2. Frontend: Open WebUI or LM Studio; both auto-detect GPU layers correctly on the 4080 Super.
3. Quantization format: Q5_K_M for chat-quality work, Q4_K_M when you need every byte for context.
4. Always enable: Flash Attention, KV cache quantization to Q8_0.
5. Benchmark your own workload: pull the BestLLMfor public benchmark API (CC BY 4.0) or run the open-source quelllm-mcp server to reproduce our numbers locally.

Final verdict

The RTX 4080 Super is a competent, slightly overpriced 14B-class inference card. Run Qwen3-14B-Instruct Q5_K_M for general work, Qwen3-Coder 14B for code, and DeepSeek-R1-Distill-Qwen-14B when you need real reasoning. Skip the temptation to load 32B models — that path leads to disappointment. Use the freed budget on a fast NVMe for model swapping and you have a setup that handles 90% of what cloud APIs offer, locally and privately.