Best local LLM for 8GB VRAM
Last updated 2026-05-26 · Page updated 2026-07-13
Top 8 open-source picks for 8 GB VRAM budgets, ranked by benchmark performance and real-world fit. Updated monthly.
Eight gigabytes is the most common VRAM budget in gaming laptops and mid-range desktop cards — and it is genuinely enough for local LLM work, provided you respect the quantization shown on each card below. Every model in this ranking fits within 8 GB at Q4_K_M according to our compatibility data, from compact 7B builds around 5 GB up to 12B-class models that sit right at the ceiling.
Two practical rules. First, prefer the Q4_K_M build unless you have measured headroom for Q5 — on this budget, fidelity gains rarely justify an out-of-memory error. Second, keep context length in check: the KV cache grows with every token, and a long prompt can push an otherwise comfortable model into system RAM, where speed collapses. If your card carries more memory, the 12 GB and 24 GB guides cover the larger models that extra headroom unlocks.
Granite 4.0 H-Tiny 7B-A1B
IBM's edge-class hybrid MoE with 7B total and only 1B active parameters — Apache 2.0 licensed and built for embedded and low-cost serving.
Mistral Nemo 12B Instruct
Mistral AI and NVIDIA's co-developed 12B instruct model with 128k context, the Tekken tokenizer, and strong European multilingual coverage.
Gemma 3 12B
The 12B sweet spot of Google's Gemma 3 line — multimodal, 128K context, and 140 languages. Fits on a single consumer GPU with room for batching.
Nemotron Nano v2 VL 12B
NVIDIA's 12.6B enterprise VLM with strong DocVQA and ChartQA scores, tuned for professional document extraction workflows.
Lucie 7B
A French-sovereign 7B model from OpenLLM-France, backed by CNRS and LINAGORA, with a fully transparent and auditable training corpus.
DeepSeek R1 Distill 7B
A 7B DeepSeek model distilled from R1 671B with explicit chain-of-thought reasoning. Surprisingly strong on AIME and MATH for its size.
Qwen 3 8B
Alibaba's 8B dense model with a toggleable thinking mode and broad multilingual coverage. Punches well above its weight for an 8B and runs comfortably on a single consumer GPU.
Qwen 2.5 VL 7B
A 7B vision-language model from Alibaba with state-of-the-art results in its class, scoring 95.7 on DocVQA. Handles hour-long video, bounding-box grounding, and multilingual OCR.
Which GPU should you buy to run Granite 4.0 H-Tiny 7B-A1B?
To run Granite 4.0 H-Tiny 7B-A1B locally at Q4, you need ~4 GB of VRAM. The best value for this is a RTX 5060 (8 GB VRAM).
As an Amazon Associate, BestLLMfor earns from qualifying purchases, at no extra cost to you. It does not influence our independent rankings.
Frequently asked questions
What is the best local LLM for 8 GB VRAM budgets?
Granite 4.0 H-Tiny 7B-A1B tops this ranking — a 7B model, licensed under Apache 2.0, needing about 4 GB of VRAM at Q4 quantization. See the full list below for the runner-ups and how they compare.
How much VRAM do I need to run Granite 4.0 H-Tiny 7B-A1B?
At Q4 quantization, Granite 4.0 H-Tiny 7B-A1B needs about 4 GB of VRAM and fits comfortably on a single 24 GB GPU.
Which of these models fit an 8 GB GPU?
At Q4 quantization, Granite 4.0 H-Tiny 7B-A1B, Mistral Nemo 12B Instruct, Gemma 3 12B, Nemotron Nano v2 VL 12B, Lucie 7B and 3 more fit within 8 GB of VRAM.
Are the models on this 8 GB VRAM budgets list free for commercial use?
Licenses across this list include Apache 2.0, Gemma, MIT, NVIDIA Open Model License. Check the specific license of each model on its catalog page before deploying commercially, as terms vary by author.
What context window do these models support?
Context windows on this list range from 4k to 128k tokens, depending on the model.