Which LLM on RTX 5070 (12 GB) ?

#RTX 5070 for a local LLM: what 12 GB of GDDR7 enables

A RTX 5070 easily runs 4B to 12B models in Q4. Qwen 3.5 9B weighs 6.6 GB and Gemma 4 12B weighs 7.7 to 8 GB according to the Ollama library: both fit within 12 GB, leaving 4 to 5 GB of headroom for context, allowing 16,000 tokens or more with the quantized K/V cache. However, the card cannot load any 14 GB model: Mistral Small 24B and gpt-oss 20B exceed 12 GB. Its speed is excellent for interactive use: 672 GB/s on a 192-bit bus. The 5070 is therefore suitable for everyday use, but hits a clear ceiling as soon as you target higher-quality models.

Architecture

Blackwell, 6,144 CUDA cores and 5th-generation Tensor Cores according to NVIDIA.

Memory

12 GB of GDDR7 on a 192-bit bus, with 672 GB/s of bandwidth according to Wikipedia.

Power

250 W total graphics power; NVIDIA specifies a 650 W minimum power supply for the entire PC.

Launch price

549 dollars, released on March 5, 2025, according to Wikipedia.

#Which models fit in 12 GB

The site's rule of thumb is about 0.6 GB per billion parameters in Q4, for weights alone. With 12 GB, that puts the practical limit around 14 to 16 billion parameters if the context stays short, and around 12 billion if you want to work with long documents. For an 8B to 9B model, the headroom is far greater than what is needed for a typical context.

#Install Ollama and check the card

1. 01
   NVIDIA driver

   Ollama requires a NVIDIA driver version 550 or later. For an RTX 50, install the latest driver offered by NVIDIA and check with nvidia-smi.
2. 02
   Install Ollama

   Download the installer for your system: CUDA is included.
3. 03
   Run a model

   Try ollama run gemma4:12b to test the card’s limit, or ollama run qwen3.5:9b for a lighter model.
4. 04
   Check

   Run ollama ps: the PROCESSOR column must display 100% GPU.

#What speed to expect: a ceiling, not a measurement

No throughput figure presented here is an in-house measurement. The generation ceiling is bandwidth divided by weight size. A public third-party measurement (LLaMA 3 8B in Q4_K_M under llama.cpp, May 2024) records 106 tokens/s on a RTX 4080 with a ceiling of 156 tokens/s, or about 68%. Using 70% as a rough benchmark, we get the following estimates for a short context.

These ceilings are far above human reading speed: the 5070 will never be slow on these models. It is limited by capacity, not speed. Estimates drop with context, and prompt processing weighs more heavily on compute than on memory.

#Manage context on 12 GB

The 4 GB headroom of Gemma 4 12B disappears quickly with a long context. Ollama quantizes the K/V cache to q8_0 to use about half the memory of the default f16, with very little loss of precision according to its documentation; this requires Flash Attention. This is the setting that makes a 12B model usable on long documents.

export OLLAMA_FLASH_ATTENTION=1
export OLLAMA_KV_CACHE_TYPE=q8_0
export OLLAMA_CONTEXT_LENGTH=16384
ollama serve

One tricky point if you serve multiple people: multiple parallel requests increase the reserved context by the same factor, according to the Ollama FAQ. A model that fits one user may overflow with three.

• Quantize the KV cache to save VRAM
• Understanding the context window

#What you can actually do with 12 GB

Twelve gigabytes changes the nature of the workloads compared with 8 GB. A 9B model leaves enough room for a 16,000-token context and a small embeddings model: this is a comfortable personal RAG setup. A 12B model provides higher-quality writing and summarization responses, at the cost of a more limited context. A coding assistant can make do with an 8B to 12B model and targeted excerpts, but not an entire repository loaded at once.

Choosing between 9B and 12B depends on the task. The 9B leaves more headroom and works well with long contexts; the 12B is more capable, but every gigabyte of context counts. Start with the smallest model that meets your needs, and move to the 12B only if quality requires it.

#When the 5070 is no longer enough

Three signals indicate that you need more memory. You want a 14B to 24B model, such as Mistral Small 24B or gpt-oss 20B. You need a context longer than 16,000 tokens on a 12B model. You load several models together: generation, embeddings, and reranking. In these cases, the 5070’s speed is not the issue, and the 5070 Ti, 5080, or a 24 GB card are the next steps up.

#What 12 GB is not enough to load

A model that spills over does not crash: some of the weights are read from system RAM, and throughput drops sharply because the PCIe link is much slower than GDDR7. A 14 GB model on 12 GB is therefore usable for testing, but not for daily use. For these models, aim for 16 GB or more.

#Game and run an LLM on the same card

RTX 5070 is often used for both purposes. The loaded model occupies VRAM as long as it remains in memory, and a recent game also needs several gigabytes: on 12 GB, the two quickly exceed the limit together. The rule is simple: unload the model before launching a game, then reload it afterward. Ollama frees memory after a period of inactivity, and ollama ps shows what is still loaded. If you want an assistant available while you play, choose a small 4B model.

#A used 4070 Ti Super instead?

The question comes up often: a used RTX 4070 Ti Super with 16 GB versus a new RTX 5070 with 12 GB. For an LLM, capacity wins: 16 GB can load 14 GB models that the 5070 can't fit. The 5070 retains the advantages of a warranty, lower power consumption (250 W), and GDDR7 memory. If your models fit within 12 GB, the new 5070 is a reasonable choice; otherwise, capacity comes first. The dedicated guide covers the 4070 Ti Super in detail.

• Which LLM on RTX 4070 Ti Super

#5070 or 5070 Ti: 12 GB or 16 GB

The 5070 Ti brings two things: 4 GB of additional memory, which enables Mistral Small 24B and gpt-oss 20B, and 33% more bandwidth. The recommended launch prices were 549 dollars for the 5070 and 749 dollars for the 5070 Ti, a 200-dollar difference. If your models fit within 12 GB, the 5070 is enough; if you are targeting 14 GB models, the difference is a good investment. Check the price tracker to compare prices when you buy.

• Which LLM on RTX 5070 Ti
• Which LLM for 12 GB of VRAM
• Local AI graphics card prices

#2026 verdict

Buy a 5070 if

Your models range from 4B to 12B and you want Blackwell-generation speed. It is highly capable for this use.

Choose the 5070 Ti if

Want to explore 14B to 24B models? The 16 GB makes the difference.

Watch out for the game and the LLM together

A recent game and a loaded model share the 12 GB: close one before launching the other.

In short, the 5070 is a fast card whose only drawback is stopping at 12 GB. If that limit does not bother you today, it will not bother you over the next few months either, as long as you stick to 4B to 12B models; if it does bother you, it is better to pay for the capacity now.

• Source: official NVIDIA specifications (RTX 5070 and 5070 Ti)
• Source: official Ollama FAQ (Flash Attention, K/V cache, parallelism)
• Source: Gemma 4 sizes in the Ollama library

#Frequently asked questions