Which LLM on Radeon RX 9060 XT (8 / 16 GB) ?

#RX 9060 XT: the 16 GB version is the only serious option for LLMs

To run LLMs locally, choose the Radeon RX 9060 XT in the 16 GB version. Both versions share the same GPU and the same 320 GB/s bandwidth; only capacity changes, and that is what determines what you can load. With 8 GB, you are limited to 7- to 9-billion-parameter models in Q4, with a short context. With 16 GB, you can move to 12- to 24-billion-parameter models, a longer context, and more faithful quantization. At launch, AMD listed $299 for the 8 GB version and $349 for the 16 GB version: a $50 difference to double the memory.

Prices change; to find today's price and the price per gigabyte of VRAM, check the local AI graphics card price tracker. The key principle: on an inexpensive card, the premium for the 16 GB version is small compared with what it unlocks.

#ROCm, Vulkan, and Windows compatibility

The RX 9060 XT appears in the ROCm 7.2.1 compatibility matrix for Radeon cards, for Ubuntu 22.04, 24.04, and 25.10. Ollama also lists it among the cards supported by ROCm on Linux, with the ROCm v7 driver. On Linux, installing Ollama with ROCm therefore follows the same path as with other Radeon cards: the Ollama guide with AMD GPUs details the process.

Be cautious on Windows. The list of cards that the Ollama documentation gives for ROCm on Windows stops at RX 7000 and does not mention RX 9000. Ollama also states that Vulkan provides additional support on Windows and Linux, enabled by default. A RX 9060 XT may therefore work on Windows through Vulkan; check with ollama ps that the model is actually on the GPU and not the CPU. LM Studio and llama.cpp also offer a Vulkan backend.

#What fits in 8 GB and 16 GB

The weights below come from the QuelLLM catalog, in Q4 unless otherwise noted. In addition to these weights, account for the KV cache, which grows with the context, and roughly one gigabyte of headroom for the system and buffers. A model that fills the card to 100% with weights is therefore too large.

The practical conclusion fits in one sentence: 16 GB opens up the 12- to 24-billion-parameter category, where you’ll find significantly more capable models for reasoning, code, and French. The guide to 16 GB of VRAM provides the complete list; the 8 GB guide covers tricks for stretching memory.

#Expected speed: bandwidth sets the ceiling

Text generation reads most of the model's weights at each token, so maximum speed is bandwidth divided by the size of the weights read. At 320 GB/s, that's a ceiling of about 53 tokens per second for a Qwen 3.5 9B in Q4 (6 GB), 46 for a Gemma 4 12B (7 GB), 23 for a Mistral Small 24B (14 GB), and about 32 for the same Qwen 3.5 9B in Q8 (10 GB). These are theoretical ceilings, never reached exactly; MoE models, which read only their active weights, partly avoid this limitation.

The only comparable public benchmark is the llama.cpp community’s ROCm table: for the RX 9060 XT, one participant reports 1 420 tokens per second for prompt processing and 68 for generation on a Llama 2 7B in Q4_0, without Flash Attention. This small model is much lighter than those in the previous table. It provides an order of magnitude, not the speed of a 12B or 24B on your setup, which will vary with the driver, operating system, and manufacturer’s card.

#Set Ollama so you don't exceed 16 GB

On 16 GB, every gigabyte matters. Three settings prevent silent overflow onto the CPU, which causes a speed drop without an error message. The first is context size: Ollama’s documentation specifies a default window of 4,096 tokens, adjustable with OLLAMA_CONTEXT_LENGTH; each doubling of the context increases the KV cache. The second is the KV cache itself, quantifiable with OLLAMA_KV_CACHE_TYPE (f16 by default, q8_0 to reduce it) when Flash Attention is active. The third is monitoring: after a response, ollama ps displays the split between GPU and CPU.

OLLAMA_CONTEXT_LENGTH=16384 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve
# dans un autre terminal
ollama run <nom-du-modèle>
ollama ps

One example of the reasoning: a 14 GB model such as Mistral Small 24B in Q4 on a 16 GB card leaves about 2 GB for context, cache, and buffers. If ollama ps shows a GPU/CPU split, reduce the context first, then quantize the KV cache, then drop down one quantization level, rather than letting the model spill over. The guide to KV cache quantization explains the process in detail.

#Buy, wait, or target 24 GB

Three situations arise. If you already have a recent desktop PC and a 450 W or higher power supply, the 9060 XT 16 GB is a straightforward way to move from limited use with small models to serious use for chat, translation, and light coding. If you are targeting models with 27 to 35 billion parameters, neither 16 GB nor its 320 GB/s is sufficient: look at 24 GB cards, covered in the RX 7900 XTX guide. If you cannot decide based on price, compare the cost per gigabyte of VRAM in the site’s tracker, which is the right criterion for local AI.

#Two RX 9060 XT: 32 GB for less?

The question keeps coming up in searches. In llama.cpp, the default distribution mode splits the model by layers across the cards in a pipeline, and an option sets the proportion assigned to each card. Two 9060 XT 16 GB cards provide 32 GB of total VRAM at about 320 W of typical power according to AMD, which is enough to load a Qwen 3.8 27B in Q4 (16 GB of weights) with plenty of context, or a 30- to 35-billion-parameter MoE (19 to 21 GB).

Two limitations to know. Per-token speed doesn't double: the cards work one after another on each token, and each retains its 320 GB/s of bandwidth. The motherboard must also provide two usable PCIe slots and a well-ventilated case. A single 24 GB card, such as the 7900 XTX, offers 960 GB/s over a single bus; for a model that fits on it, it generates noticeably faster.

#FP8: an advantage on paper, few consequences with Ollama

The Radeon RX 9000 cards use the RDNA 4 generation, and AMD lists 205 TFLOPs of FP8 compute for the 9060 XT, which the RX 7900 XTX specifications do not mention. For typical use with Ollama or LM Studio, this matters little: GGUF models are quantized as integers (Q4, Q5, Q8), not FP8. It becomes useful with engines that use FP8 weights, such as vLLM, whose documentation lists the RX 9000. Do not use it as a purchasing argument if you are sticking with Ollama.

#RX 9060 XT or RTX 5060 Ti: the 16 GB showdown

NVIDIA offers the RTX 5060 Ti with 16 GB and 8 GB, using GDDR7 on a 128-bit bus, versus GDDR6 on 128 bits for the AMD card. The faster memory gives NVIDIA the bandwidth advantage, and therefore higher generation speed; see the NVIDIA spec sheet for the exact throughput. The other advantage is software: CUDA is the default target for most AI projects.

• Which LLM for 16 GB of VRAM: the complete list
• Which LLM for 8 GB of VRAM: tips and limitations
• Ollama with AMD GPUs (ROCm): step-by-step installation
• LLM on RTX 5060 Ti (8 / 16 GB)
• LLM on RX 7900 XTX (24 GB)
• Local AI graphics card prices

• Source: AMD specifications for the RX 9060 XT (16 GB)
• Source: Ollama GPU documentation
• Source: ROCm compatibility on Radeon
• Source: llama.cpp performance table on ROCm

#Frequently asked questions