Guide · 2026-08-29

What LLM Can You Run on 8GB VRAM in 2026?

Last updated 2026-08-29

8GB cards like the RTX 4060 and RX 7600 handle 8B-9B models comfortably. Here's exactly which models, quant levels, and settings fit — and when to upgrade.

By Mohamed Meguedmi · 7 min read

8GB of VRAM Is the Mainstream Floor for Local LLMs

If you're shopping for a GPU to run large language models on your own machine, 8GB of VRAM is the number you'll see most often in 2026 — it's what ships in the majority of mainstream gaming and workstation cards sold in the US over the last few years. The good news is that 8GB is a genuinely usable amount of memory for local inference, not just a bare minimum you'll immediately outgrow. It comfortably runs the 8B-9B model class in Q4 to Q6 quantization without any of the layer offloading or CPU spillover that tanks throughput.

Qwen 3.5 9B at Q4 quantization comes in at 6.6GB, and with a 16K token context window added on top, it still fits inside an 8GB budget with a reasonable margin. Granite 4.2 8B at Q5 quantization is even lighter, at 5.3GB, leaving noticeably more headroom for your browser, IDE, or whatever else is fighting for VRAM in the background. Treat this as the comfortable baseline for regular, day-to-day local LLM use — not the ceiling, but not a compromise either.

Which GPUs Actually Ship With 8GB of VRAM

Before you plan around an 8GB budget, it's worth knowing exactly which cards fall into that bucket, since VRAM capacity — not raw compute — is almost always the limiting factor for local LLM work.

• Nvidia (8GB): RTX 4060, RTX 5050, RTX 5060, RTX 5060 Ti 8GB, RTX 4060 Ti 8GB, RTX 3070, RTX 3070 Ti, RTX 3060 Ti, RTX 3050 8GB, RTX 2080, RTX 2080 Super, RTX 2070, RTX 2060 Super.
• AMD (8GB): RX 7600, RX 7600 XT, RX 6600 XT, RX 6650 XT, RX 5700 XT.
• Apple unified memory (8GB): MacBook Air M1/M2 with 8GB unified memory — in practice, only about 5GB of that is realistically available to the GPU once the OS and background processes take their share.

Notice that this list spans four Nvidia GPU generations (Turing through Blackwell) and two AMD generations. Older cards like the RTX 2060 Super or RX 5700 XT can still run the same 8B-9B models as a current-generation RTX 5060 — VRAM capacity, not architecture age, is what determines whether a model fits. Compute speed will differ, but capacity is capacity. For a full breakdown of how these cards stack up against each other, see our GPU comparison hub.

The 8B-9B Sweet Spot: What Actually Fits

Here's how the numbers break down for the models that fit comfortably on an 8GB card, plus the one that only just makes it in.

Model	Quantization	VRAM Footprint	Context	Fit
Qwen 3.5 9B	Q4	6.6GB	16K	Comfortable
Granite 4.2 8B	Q5	5.3GB	Default	Plenty of margin
Granite 4.2 8B	Q5 + KV cache Q8	~6.5GB	16K	Comfortable, extended context
Gemma 4 12B	Q4	7.6GB	Reduced	Tight — just fits

The pattern is consistent: 8GB comfortably absorbs anything in the 8B-9B range, and it can stretch to a 12B model like Gemma 4 if you're willing to run a smaller context window in exchange. What it can't do is run a 14B model at any reasonable context length — more on that below. For a curated shortlist of what to install first, check our best models for 8GB VRAM guide.

Why 14B Models Don't Fit — the 12GB Wall

A 14B model at Q4 quantization needs roughly 8.5GB just for the weights, which already overshoots an 8GB card before you've loaded a single token of context or left any headroom for the OS. Gemma 4 12B at Q4 (7.6GB) is the closest you can get to that next tier — it does load and run on 8GB, but only at the cost of a meaningfully reduced context window, which limits how much document or conversation history you can feed it in one pass.

If you want 14B-and-up models running with a full-size context and no compromises, 12GB is the real frontier in 2026 — not 8GB. That's a hard-budget line, not a soft recommendation: the jump from 8GB to 12GB is what unlocks the next model tier cleanly, rather than forcing you into reduced-context workarounds. Our best models for 12GB VRAM guide covers what opens up once you cross that line.

Tuning Tricks to Stretch an 8GB Card Further

None of these turn an 8GB card into a 12GB one, but they buy real, measurable headroom for context length and quality without touching your hardware budget.

• Quantize the KV cache to Q8: Setting OLLAMA_KV_CACHE_TYPE=q8_0 shrinks the memory the context window itself consumes. On Granite 4.2 8B, this is what makes a full 16K context possible in around 6.5GB total — without it, the same context length would push past what an 8GB card can hold alongside the model weights.
• Leave Flash Attention on: It's enabled by default on Turing and newer Nvidia GPUs (RTX 20-series and up), and it delivers a 10-15% throughput gain on longer contexts at effectively no cost. There's no reason to disable it on any of the cards listed above.
• Choose Q5 or Q6 over Q4: On an 8GB card, Q5 (roughly 5GB) or Q6 (roughly 6GB) quantization is well within budget, and both preserve noticeably more output quality than Q4 (roughly 4.4GB). Dropping to Q4 on an 8GB card is usually leaving quality on the table for VRAM headroom you don't actually need.
• Skip multi-stage RAG pipelines: Running an embedding model, a reranker, and the LLM together typically adds up to 9-10GB — tight enough that it won't reliably fit alongside an 8B-9B model on an 8GB card. If your workflow depends on retrieval-augmented generation with a reranking stage, that's a strong signal you need more VRAM, not just better tuning.

When (and What) to Upgrade

If you're running into the 14B ceiling regularly, or a multi-stage RAG pipeline is what you actually need, here's where the next real jump in capability comes from and what it costs.

Upgrade path	Typical US price	What it unlocks
RTX 5060 Ti 16GB	~$429 MSRP new	Roughly double the VRAM; access to Mistral Small 24B, gpt-oss 20B, and Devstral 24B — the single biggest capability jump on this list
RTX 4060 Ti 16GB (used)	~$350-380	Same 16GB capability tier as the 5060 Ti, at a used-market discount
RTX 3060 12GB (used)	~$200-230	+50% VRAM over an 8GB card, for less than the price of buying an 8GB card new — a budget-conscious way to clear the 12GB wall above

The RTX 5060 Ti 16GB is the upgrade worth prioritizing if the budget allows it: doubling VRAM from 8GB to 16GB doesn't just make more of the same models run — it opens an entirely new class of 20B-24B models that simply cannot run on an 8GB card at any quantization level. See how the 16GB and 12GB tiers compare directly in our RTX 3060 12GB vs. RTX 5060 Ti 16GB comparison.

Verdict: Is 8GB Enough in 2026?

For anyone running 8B-9B models as a daily driver — coding assistance, drafting, summarization, general chat — 8GB VRAM is a genuinely solid, comfortable tier in 2026, not a bottleneck you're constantly fighting. Where it falls short is anything that needs a 14B+ model at full context, or a RAG pipeline with a dedicated reranker. Those workloads need 12GB or more, and no amount of KV cache tuning changes that math.

If you're deciding between a new 8GB card, a used 12GB card, or holding out for a 16GB card, run the numbers for your specific model shortlist through our hardware configurator before you buy — it accounts for quantization, context length, and OS overhead rather than just raw model size. All of the VRAM-fit figures referenced on this page are also available programmatically through the BestLLMfor public API (CC BY 4.0) and our open-source MCP server, if you'd rather script your own capacity checks against a model list.