Which LLM on RTX 2060 / 2060 Super (6–8 GB) ?

#RTX 2060, 2060 Super, 2060 12 GB: three cards, three use cases

The RTX 2060 was released on January 15, 2019, the 2060 Super on July 9, 2019, and a 12 GB variant of the 2060 on December 7, 2021, according to Wikipedia's GeForce 20-series table. All are based on the Turing TU106 chip. For an LLM, what matters is memory (capacity, bus, bandwidth), not CUDA cores.

The 2060 Super offers 448 GB/s of bandwidth versus 336 GB/s for the original 2060; with the same model, it generates about one-third faster. The 12 GB variant is rare and arrived late, but it's by far the most interesting for local AI: it's the only one of the three that lets a 14B model in Q4 (about 9 GB) fit with headroom. So check the card's exact capacity before buying: the name alone doesn't tell you.

#What each variant can run

The rule is the same as for any card: Q4 weights (the site's reference point: 3B ≈ 2 GB, 7-8B ≈ 5 GB, 14B ≈ 9 GB), plus the context cache and about 0.5 GB reserved by the system and engine. The table also gives the theoretical generation ceiling, which is bandwidth divided by weight size. We have no throughput measurements for these cards and do not cite any.

These ceilings are never reached in practice; they are used to compare cards with one another and determine whether a model will be interactive. A model that weighs twice as much generates at roughly half the speed. For conversational use, anything above around ten tokens per second in real-world performance is comfortable to read.

#Turing in 2026: still supported

The card’s age is less concerning than you might think. Ollama's documentation states that it supports NVIDIA GPUs with compute capability 5.0 and higher with driver 550 or newer, and lists the RTX 2060 in the 7.5 capability family. CUDA 13 release notes, meanwhile, state that support for Maxwell, Pascal, and Volta was removed from some libraries: Turing is the oldest generation that remains on this path. This is an argument for the 2060 over GTX 10 cards in the long term, even though Ollama still supports capabilities 5.0 through 6.2 with driver 570 or newer: the risk is that future features will target only newer architectures.

Text generation on this card is limited not by compute power but by memory bandwidth, so do not pay a premium for Tensor Cores alone. The driver is the real prerequisite: an up-to-date installation avoids most GPU detection failures.

#The memory calculation for your card

The calculation takes three lines: card memory, minus about 0.5 GB reserved by the system and engine, minus the model size, equals the space remaining for the context cache. What the cache consumes per token depends on the model architecture; the site's calculator gives the figure for a specific model, and q8_0 cache quantization cuts it roughly in half.

This table explains why the same model family is pleasant on the Super and painful on the 6 GB version: with less than a gigabyte of headroom, a few thousand context tokens are enough to push the model onto the processor. It also shows that the 14B on 12 GB is real but tight: 2.2 GB for context, which requires a short window.

• Memory calculator for a model and context

#Tips for the 6 GB and 8 GB versions

For the 6 GB card, the guide “Which LLM for 6 GB of VRAM” breaks down the complete memory budget; in short, target a 4B model or an 8B model with a short context. On the 8 GB Super, context becomes the first setting to monitor.

1. 01
   Install a recent driver

   Update the NVIDIA driver to version 550 or later, as required by Ollama, then install Ollama for your system.
2. 02
   Choose the model size

   6 GB: a 4B, or an 8B with a short context. 8 GB: an 8B in Q4 with a context of 4 000 to 8 000 tokens. 12 GB: a 14B in Q4, with a short context.
3. 03
   Reduce the context cache

   Enable Flash Attention and q8_0 cache quantization: according to the Ollama FAQ, it cuts cache memory usage by about half compared with f16.
4. 04
   Check with ollama ps

   The Processor column should show 100% GPU. A split between CPU and GPU indicates offloading, which means lower speed.
5. 05
   Close resource-hungry applications

   A browser with hardware acceleration or a game uses VRAM that the model cannot use.

OLLAMA_FLASH_ATTENTION=1
OLLAMA_KV_CACHE_TYPE=q8_0
OLLAMA_CONTEXT_LENGTH=6144

The Ollama FAQ also offers a more aggressive q4_0 cache quantization that further reduces memory at the possible cost of quality; test it with your workload before keeping it.

• Which LLM for 6 GB of VRAM
• Which LLM for 8 GB of VRAM?
• Quantize the KV cache

#What you can really do with a 2060

Chat and writing

A 4B or 8B responds correctly for everyday use; this is where the card delivers on its promise best.

RAG and documents

A small 4B model and an embedding model fit in memory; limit the number of excerpts you send.

Code

Autocompletion with a small code model; no reliable agent at this size.

What exceeds the card's capacity

Large contexts, heavy vision workloads, 14B models on 6 or 8 GB, and any agent that chains tools with long prompts.

The 2060 is also a 160 to 185 W card: under sustained load, it gets hot and loud like any card in its class. The thermal guide explains how to cap power for a server that runs all day.

• Optimize thermals during inference

#Verdict and when to move on

An already-installed RTX 2060 does not need replacing if you use it for chat, text summarization, and light RAG. If you are buying, memory takes priority over everything else: the 12 GB variant or the 2060 Super 8 GB are preferable to the 6 GB version. Prices change every week and are not listed on this page; the site’s tracker records the lowest price for each card.

This comparison corrects a common misconception: the RTX 3060 12 GB is not faster than the 2060 Super for text generation. Its bandwidth is 360 GB/s according to Wikipedia, versus 448 GB/s for the 2060 Super. It wins on capacity: 12 GB can load models that the Super cannot accommodate. Choose based on the model you want to run, not on a general speed percentage.

If you buy a used card, check three things before keeping it. Use nvidia-smi to verify the advertised total memory: the card name alone is not enough to distinguish the 6, 8, and 12 GB variants. Run a continuous generation for about twenty minutes and monitor the temperature and core frequency: a card that slows down significantly needs cleaning or new thermal paste. Finally, test a real model with the intended context and verify with ollama ps that it remains entirely on the card.

• Which LLM on RTX 3060 12 GB
• Which LLM on RTX 2070 / 2070 Super
• Which LLM on RTX 3050
• Local AI graphics card price tracking
• Source: Wikipedia, GeForce 20 Series
• Source: Ollama, supported hardware
• Source: official Ollama FAQ

#Frequently asked questions