Which LLM on Radeon RX 9070 (16 GB) ?

#What a RX 9070 does in 2026

The RX 9070 handles 8- to 14-billion-parameter models in Q4 without difficulty and accepts 20B to 24B models, with a short context for the largest ones. Its 16 GB of GDDR6 at 20.1 Gb/s on a 256-bit bus gives it 640 GB/s of bandwidth, correcting the previous version of this page (644). On Llama 2 7B in Q4_0, the public discussion in the llama.cpp repository reports 119.71 tokens per second for generation under Vulkan and 114.48 under ROCm, with prompt processing at 3,164 tokens per second under Vulkan, faster than the RX 7800 XT and 7900 GRE. Its other advantage is its 220 W TDP, 40 to 50 W lower than its older 16 GB counterparts.

Architecture

RDNA 4, a chip in the RX 9000 series, on the same 256-bit bus as the 9070 XT.

Compute

3,584 stream processors (4,096 for the 9070 XT).

Memory

16 GB GDDR6, 20.1 Gb/s, 256-bit bus, 640 GB/s.

Power consumption

220 W TDP (304 W for the 9070 XT).

Pricing

Not specified: it changes every week; see /prix-gpu-ia.

#Linux, Windows: ROCm or Vulkan

This is the distinctive feature of RDNA 4 generation: support differs by operating system. AMD lists the RX 9070 in ROCm (gfx1201 target), and Ollama documentation cites it for Linux with the ROCm v7 driver. For Windows, however, Ollama's ROCm list stops at RX 7900 XTX; RX 9070 is not included. It remains usable through Vulkan, enabled by default, which also delivers good throughput. AMD's documentation finally specifies that Radeon RX 9000 are supported only on Ubuntu 24.04.4, Ubuntu 22.04.5, RHEL 10.1, and RHEL 9.7.

#What fits in 16 GB of VRAM

Allow about 15 GB for the model and its KV cache when the card is not driving the display. The QuelLLM catalog lists the Q4 weights below; the context is added on top.

These 16 GB are the boundary between the 24B, which fits, and the 26 to 30B models, which no longer fit. At this precise boundary, 16 GB cards from all brands are equivalent: the VRAM-by-page comparison covers them. What sets the RX 9070 apart is what it delivers beyond this capacity: faster prompt processing and lower power consumption.

• Which LLM for 16 GB of VRAM, regardless of card

#Measured throughput: Vulkan ahead of ROCm

The figures come from public discussions in the llama.cpp repository, all using Llama 2 7B in Q4_0 (3.56 GiB) and llama-bench; they are not measurements from the site. On the RX 9070, Vulkan delivers 3,164.10 tokens per second for prompt processing and 119.71 for generation; with Flash Attention, 2,859.98 and 119.51. ROCm delivers 2,381.77 and 114.48. The two backends are close for generation, with Vulkan slightly ahead, and the gap is more pronounced for prompt processing. Configurations, drivers, and commits differ from one series to another: treat these gaps as orders of magnitude.

The theoretical generation ceiling is 640 GB/s divided by 3.82 GB, or approximately 167 tokens per second; the card reaches 71% of that under Vulkan. Applying this throughput to larger models gives rough estimates. These are calculations, not measurements, and they assume that a larger model behaves like the reference 7B.

#What RDNA 4 changes for a local LLM

RDNA 4 adds dedicated AI hardware: the RX 9070 includes 112 AI accelerators according to the series spec sheet, versus 128 for the 9070 XT. You might expect much higher generation throughput than RDNA 3. Public benchmarks do not show that: 119.71 tokens per second for the RX 9070 under Vulkan, versus 118.27 for the RX 7800 XT in the same discussion. LLM generation is limited by memory bandwidth, and 640 GB/s is not far from the 624 GB/s of the 7800 XT. AI accelerators help more with prompt processing, where the RX 9070 is 57% faster than the 7800 XT.

The consequence for buyers: don't pay for RDNA 4 for conversational speed; pay for it to process prompts, consume less power, and extend software support, which will last longer for newer cards than for older ones. If your use case is chatting with an 8B to 14B model, a used RX 7800 XT delivers similar generation results.

#Compared with RX 9070 XT: the gap is larger than “10%”

The previous version of this page claimed that a RX 9070 was 10 to 12% slower than the 9070 XT. On the reference 7B model under Vulkan, the 9070 XT generates at 137.11 tokens per second versus 119.71, or 15% faster, and processes prompts at 5,036 tokens per second versus 3,164, or 59% faster. Yet both cards have the same memory, 16 GB, and the same bandwidth according to their specifications. The difference therefore comes from compute (4,096 stream processors versus 3,584), which has little impact on generation and a major impact on prompt processing.

In plain terms: for chat, the 9070 loses 15% speed in exchange for 84 W less power, a reasonable tradeoff. For RAG and long documents, the 9070 XT reads about 1.6 times faster.

• Radeon RX 9070 XT: 16 GB and more compute
• Radeon RX 9070 XT hardware specifications

#Against 16 GB RTX cards: the prompt remains the weak point

Compared with the NVIDIA 16 GB cards measured under Vulkan, the RX 9070 generates slightly more slowly, 8% below the RTX 4070 Ti Super and 12% below the RTX 5070 Ti, and reads the prompt at roughly half the speed. The RTX 5070 12 GB, often cited as a direct competitor, does not appear in the reference Vulkan discussion: there is therefore no reliable numerical comparison with it, only the capacity difference, 16 GB versus 12 GB.

• RTX 5070 Ti: 16 GB on the NVIDIA side
• RTX 5070: the 12 GB competitor

#Getting started

1. 01
   Choose the system

   On Linux, use Ubuntu 24.04.4, Ubuntu 22.04.5, RHEL 10.1, or RHEL 9.7 to stay within AMD's support scope. On Windows, rely on Vulkan.
2. 02
   Install Ollama

   On Linux, install the ROCm v7 driver with amdgpu-install, then Ollama. On Windows, install Ollama: Vulkan is enabled by default.
3. 03
   Load a model

   Choose a 12B or 20B before a 24B, then verify with ollama ps that the model is running 100% on the GPU.
4. 04
   Compare the backends

   Run llama-bench with Vulkan and then ROCm on your model: in public benchmarks, Vulkan is slightly ahead.

ollama ps
rocminfo | grep -i gfx
./llama-bench -m llama-2-7b.Q4_0.gguf -ngl 100 -fa 0,1

• Ollama and AMD GPUs: ROCm, Vulkan, and useful variables
• Compile llama.cpp with Vulkan

Should you wait for a next generation? Nothing in the cited measurements justifies waiting for 8B to 24B use cases: the limit is 16 GB of capacity, which only a 20 to 24 GB card removes. If you already know you want 30B-and-larger models, skip this card; otherwise, buying a 16 GB card is justified by what you run today, not by what will be released tomorrow.

#2026 verdict

A good choice

If you want 16 GB in a modest 220 W card and accept Vulkan on Windows.

Prefer the 9070 XT

If your prompts are long: reading is 59% faster, for 84 W more.

Prefer a 16 GB RTX

If you need CUDA or twice-as-fast prompt processing.

Prices change quickly: the site tracker records the lowest price of graphics cards for local AI every Monday and Thursday, along with the price per GB of VRAM. This last figure should guide your comparison with the RTX 5070 or the RX 9070 XT.

• Graphics card prices for local AI, updated twice a week
• Ollama documentation: supported AMD cards
• ROCm GPU compatibility, AMD documentation
• Vulkan scoreboard for the llama.cpp repository

#Frequently asked questions