Which LLM on Radeon RX 6700 XT (12 GB) ?

#What a RX 6700 XT does in 2026

A Radeon RX 6700 XT works very well as an entry-level card for a local LLM, provided you accept three facts: its 12 GB of memory can accommodate a 14-billion-parameter model in Q4 (about 9 GB of weights), but not a 24B; its reliable software path is Vulkan, through llama.cpp or Ollama, because AMD does not list it in ROCm; and its generation speed, limited by its 384 GB/s of bandwidth, is sufficient for a smooth conversation on an 8B to 12B model. On a 7B in Q4_0, a measurement published in the llama.cpp repository reports 83.88 tokens per second during generation under Vulkan, slightly faster than the RTX 3060 12 GB measured in the same discussion. The real weakness is prompt processing, which is twice as slow.

Architecture

RDNA 2, Navi 22 chip, launched on March 18, 2021 (Wikipedia page for the RX 6000 series).

Compute

2,560 stream processors, 40 compute units.

Memory

12 GB GDDR6, 192-bit bus, 384 GB/s.

Power consumption

230 W of board power.

Pricing

Not listed here: used prices change every week; check /prix-gpu-ia.

#Ollama, ROCm, or Vulkan: the path that works

The most common trap with this card is looking for a ROCm guide. The Ollama documentation lists Radeon RX cards from the 6800 to the 9070 XT for Linux: the 6700 XT is not included. On the ROCm side, AMD’s compatibility table lists only professional cards from RDNA 2, such as the PRO W6800 and V620. Ollama expands this scope with Vulkan, enabled by default when the backend is installed, and this is the path that remains open to a 6700 XT. A ROCm workaround exists—the HSA_OVERRIDE_GFX_VERSION variable forces a nearby LLVM target—but it is intended for experimentation.

#What fits in 12 GB of VRAM

The site's rule remains the same: the Q4 model weight plus the KV cache plus some headroom for the driver and display. On 12 GB, a realistic budget leaves about 11 GB for the model and its context if the card is not driving the display. The sizes below come from the QuelLLM catalog and our usual benchmarks; they are model weights only.

The KV cache grows with the conversation length, which explains why a 14B model “fits” on paper but crawls as the history gets longer. The site's VRAM calculator gives the exact total for your model and context, and the guide to KV-cache quantization explains how to reclaim one or two gigabytes. For the full range of 12 GB cards, the dedicated page compares models without limiting itself to AMD.

• VRAM calculator for a given model and context
• Which LLM for 12 GB of VRAM, across all GPUs
• Quantize the KV cache to save VRAM

#Measured throughput and bandwidth ceiling

None of the throughput figures on this page are measurements from the site. The figures come from the public llama.cpp repository discussion comparing cards under Vulkan with the same model, Llama 2 7B in Q4_0 (3.56 GiB), and the llama-bench command. Two values matter there: pp512, the prompt-reading speed for 512 tokens, and tg128, the generation speed for 128 tokens. For the RX 6700 XT, the discussion reports 1,051 tokens per second for reading and 83.88 for generation; with Flash Attention enabled, 1,011 and 81.86.

This result can be compared with a simple ceiling: during generation, each token requires the card to reread all the weights, so maximum speed is roughly bandwidth divided by model size. Here, 384 GB/s for 3.82 GB of weights gives a theoretical ceiling of 100 tokens per second; the card reaches 83% of it—a good result, better than that of several newer cards in the same discussion.

These ballpark figures assume that the performance measured on the 7B carries over to larger models, which is not guaranteed: a driver, a version of llama.cpp, or different quantization can shift the result by several points. For prompt processing, retain only the ratio between cards: that is what matters for RAG use with long documents.

#Against the RTX 3060 12 GB: generation wins, prompting loses

The usual comparison says that the RTX 3060 12 GB wins thanks to CUDA. Measurements published under Vulkan put that assumption into perspective. During generation, the RX 6700 XT outperforms the RTX 3060 in both modes; during prompt processing, the RTX 3060 performs almost twice as well. Both series come from the same discussion but different versions of llama.cpp: treat the gap as an order of magnitude, not as a definitive ranking.

The practical consequence: for chatting with a model, the two cards are comparable; for querying a long document, the RTX 3060 responds faster to the first token. CUDA also retains its ecosystem advantage, with fewer workarounds to learn. At the same used-market price, the RTX 3060 12 GB remains the safer choice, and the 6700 XT is a good deal only if it’s already in your PC or significantly cheaper.

• Which LLM on RTX 3060 12 GB, the direct competitor card

#Step-by-step setup

1. 01
   Check the Vulkan driver

   On Windows, the Radeon driver includes Vulkan, and Ollama requires no additional steps. On Linux, install Mesa's Vulkan components (RADV) or the AMD driver, as documented by Ollama.
2. 02
   Install Ollama or compile llama.cpp

   Ollama enables Vulkan by default once the backend is installed. With llama.cpp, compile with the -DGGML_VULKAN=on option.
3. 03
   Give the card access under Linux

   Add the ollama user to the render group if the card isn't detected. For Ollama to read free VRAM, run it as root or grant it the capability specified in the command below.
4. 04
   Run your first model

   Download an 8B or Gemma 4 12B in Q4_K_M, run it with the command ollama run, then check with ollama ps that the model is loaded on the GPU rather than the processor.
5. 05
   Measure on your system

   Run llama-bench with the same GGUF as the public discussion to compare your card with its results before looking for a cause of slowdown.

sudo setcap cap_perfmon+ep /usr/local/bin/ollama
# llama.cpp avec Vulkan
cmake .. -DGGML_VULKAN=on -DCMAKE_BUILD_TYPE=Release
./bin/llama-bench -m llama-2-7b.Q4_0.gguf -ngl 100 -fa 0,1

If throughput seems low under RADV, the llama.cpp discussion recommends launching with the RADV_PERFTEST=nogttspill environment variable, which fixes several performance issues. The other Vulkan settings are in the dedicated build guide.

• Compile llama.cpp with Vulkan, step by step
• Ollama and AMD GPUs: what ROCm really supports

#Limitations and when to move on

Three limitations stand out. First, memory: 12 GB rules out 20- to 32-billion-parameter models, where the quality of coding and reasoning assistants is determined. Next, prompt processing, which makes long-document sessions painful. Finally, support: a card outside the official list depends on Vulkan and llama.cpp updates being maintained, with no long-term guarantee.

• Radeon RX 7700 XT, the 12 GB RDNA 3 step, officially supported
• Radeon RX 7800 XT, 16 GB to move up to 20–24B models

#2026 verdict

It's worth it

If it is already in your machine: it is one of the few 2021 cards that can still run a 14B in Q4 with a generation yield above 80% of its theoretical ceiling.

Buy a RTX 3060 12 GB instead

If you’re starting from scratch and targeting the same budget: twice-as-fast prompt processing and zero software workarounds.

Switch cards

As soon as you want 20- to 30-billion-parameter models; prices change every week, and the /prix-gpu-ia page gives the cost per GB of VRAM.

• Graphics card prices for local AI, updated twice a week
• Documentation Ollama: supported GPUs, ROCm, and Vulkan
• Vulkan scoreboard from the llama.cpp repository (Llama 2 7B Q4_0)
• ROCm GPU compatibility, AMD documentation

#Frequently asked questions