Which LLM on RTX 3060 Ti (8 GB) ?

#RTX 3060 Ti vs RTX 3060: the summary for local AI

The RTX 3060 Ti is faster than the 3060 12 GB for generating text, but it has 4 GB less, and for local AI, capacity is the limiting factor. The llama.cpp collaborative table gives 92.2 tokens per second for the 3060 Ti, versus 75.6 for the 3060 12 GB, on a 7-billion-parameter model in Q4_0: about 22% more. This advantage comes from its 256-bit bus, which provides 448 GB/s of bandwidth versus 360 GB/s. However, with 8 GB it runs 7- to 9-billion-parameter models (Granite 4.2 8B at 4.6 GB, Qwen 3.5 9B at 6 GB) but not Gemma 4 12B (7 GB of weights), which leaves no room for context. If your target is an 8B, the Ti is the faster of the two; if you are aiming for a 12B, it is out of the running.

RTX 3060 Ti

GA104 chip, December 2020, 4,864 CUDA cores, 8 GB of GDDR6, 256-bit bus, 448 GB/s.

RTX 3060

GA106 chip, 3,584 cores, 12 GB of GDDR6 on a 192-bit bus, 360 GB/s.

For an LLM

More bandwidth means more speed; less memory means fewer models. The two effects offset each other, but capacity carries more weight.

#Models compatible with 8 GB

The card’s generation throughput is not what will limit you; space is. The llama.cpp test sees 7,838 MiB of memory on the 3060 Ti, slightly less than the nominal 8,192 MiB, and that budget is shared between the weights, context cache, and display if a monitor is connected. Allow at most 6 GB for weights to keep a context of several thousand tokens.

• Which LLM for 8 GB of VRAM? (all cards)
• VRAM calculator for a given model and context

#Installation and settings

The card has compute capability 8.6 in the Ollama list, which requires driver 550 or newer. For power, the NVIDIA specification lists 200 W for the card and recommends a 600 W system power supply, versus 550 W for the 3060: a 500 W power supply, sometimes cited, is below that recommendation.

1. 01
   Install the driver

   Install an NVIDIA 550 or newer driver and verify with nvidia-smi that the card is recognized with approximately 8 GB of memory.
2. 02
   Install Ollama

   Follow the installation guide, then launch an 8-billion-parameter model in Q4 with ollama run.
3. 03
   Enable memory settings

   Start the server with OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0. The documentation specifies that the K/V cache can be quantized when Flash Attention is enabled.
4. 04
   Monitor memory

   During a long conversation, check nvidia-smi: if memory approaches 7,800 MiB, reduce the context.

OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve

• How to install Ollama
• Quantize the KV cache: save VRAM (long context)

#Speed: generation, prompt processing, and Flash Attention

A September 2026 measurement published on the llama.cpp dashboard provides a detailed look at the 3060 Ti’s behavior, using a 7-billion-parameter model in Q4_0 at 3.56 GiB. The 448 GB/s bandwidth would allow about 117 tokens per second at maximum; the measurement reaches 92 tokens, or about 79% of the ceiling. The most interesting result concerns Flash Attention.

Generation improves little, but reading a long prompt gets more than one-third faster, which matters for RAG and document summaries. At this rate, a prompt of 10,000 tokens is read in about four seconds, a theoretical calculation that assumes a constant throughput. Ollama automatically enables Flash Attention when the card supports it; you can force it with the OLLAMA_FLASH_ATTENTION=1 variable.

For a model larger than the test file, the rule of three gives rough figures: 92 × 3,82 ÷ 4,6, or about 76 tokens per second, for Granite 4.2 8B in Q4, and about 59 for Qwen 3.5 9B (6 GB). These are high theoretical upper bounds, not measurements. If you want to compare your own card with these values, the llama-bench tool from llama.cpp reproduces the test: same file Llama 2 7B in Q4_0, with all layers placed on the GPU, with and without Flash Attention. Results vary by driver, system, and card manufacturer, even with the same chip, so a difference of a few percent is nothing unusual. These speeds exceed reading speed: the card is never the limiting factor in a conversation.

#Documents and RAG: where the time goes

When you query your documents, the card first spends time reading the prompt, then generating the response. On the 3060 Ti, a 4,096-token prompt—about ten pages—takes roughly 2.2 seconds to read without Flash Attention and 1.6 seconds with it, according to speeds measured by llama.cpp. The 400-token response then takes about four to five seconds at 92 tokens per second on the test model, and longer on a current 8B. These are theoretical calculations that assume constant throughput, but they show that the wait remains a few seconds.

The RAG limit on 8 GB is therefore not time but memory: a context of several tens of thousands of tokens fills the context cache of an 8-billion-parameter model. Two ways to stay within the budget: reduce the number of excerpts injected into the prompt, and quantize the K/V cache to q8_0, which uses approximately half the memory of the default format according to Ollama's documentation.

#When to move on

Read this table from top to bottom: the 3060 Ti covers the first two rows, while the next three require more memory. If your needs fall in the lower half, changing cards is justified; otherwise, the 3060 Ti gets the job done. A higher-quality but larger model can sometimes be better than a faster card: at 92 tokens per second, speed is not your bottleneck.

#3060 Ti and 3060 12 GB: the false twin

The first instinct is to choose the faster one. However, for the 8- to 9-billion-parameter models that both cards can load, the extra 22% speed does not change the experience: in both cases, text appears faster than you can read it. By contrast, the 3060 12 GB opens up an entire category of models. It is therefore the better choice for local AI, unless your use is permanently limited to 8- to 9B models. One buying precaution: NVIDIA also sold a 3060 Ti with GDDR6X memory, distinguished by its specification sheet; the 8 GB capacity remains the same.

• Which LLM for RTX 3060 with 12 GB?
• Local AI graphics card prices (weekly tracking)

#3060 Ti and RTX 4060 Ti 8 GB

The newer RTX 4060 Ti 8 GB measures 63.9 tokens per second on the same test, significantly less than the 3060 Ti. The reason is its 128-bit bus, which limits its bandwidth despite a more efficient architecture. For text generation, the 3060 Ti therefore remains faster than this newer card at the same capacity. Its strengths lie elsewhere: newer architectural features and, according to the manufacturers’ specifications, lower power consumption, which is useful in a compact case or on a machine that stays on to serve a model. This is not a text-generation speed improvement, nor is it a memory increase. If you are choosing between the two, look at the 16 GB version of the 4060 Ti, which offers twice the memory.

• Which LLM on RTX 4060 Ti (8 / 16 GB)?

#2026 verdict

Keep it if

It works, and you use 7- to 9-billion-parameter models: speed is more than sufficient, and you should not expect any gain from switching at the same capacity.

Don't buy for AI if 12 GB is available

A 3060 12 GB opens up 12-billion-parameter models. The used-price gap changes every week: check the site's tracker.

Move to 12 or 16 GB if

You want a 12B model, a long context, or RAG over large documents.

#Frequently asked questions

• Source: llama.cpp performance table on CUDA
• Source: NVIDIA and 3060 Ti RTX 3060 spec sheet
• Source: maps NVIDIA supported by Ollama
• Source: Ollama FAQ (Flash Attention, K/V cache)