Which LLM on Radeon RX 7800 XT (16 GB) ?

#Why the RX 7800 XT is a good choice for a local LLM

The RX 7800 XT combines three rare strengths at this price point. Its 16 GB of GDDR6 on a 256-bit bus gives it 624 GB/s of bandwidth, the factor that limits an LLM's generation speed. AMD officially lists it in ROCm (RDNA 3, gfx1101 target), and Ollama cites it for both Linux and Windows. Finally, public measurements place it at 118.27 tokens per second when generating on a 7B model in Q4_0 under Vulkan—faster than a RX 7900 GRE in the same series of measurements. Its drawback compared with NVIDIA is not generation speed but prompt processing, which is three times slower than a RTX 4070 Ti Super.

Architecture

RDNA 3, Navi 32 in chiplets: one GCD and four MCDs, launched on September 6, 2023. It is therefore not monolithic.

Compute

3,840 stream processors, 60 compute units.

Memory

16 GB GDDR6, 256-bit bus, 624 GB/s.

Power consumption

263 W of board power.

Pricing

Not set here: see /prix-gpu-ia, which records the lowest price every Monday and Thursday.

#Evaluating an offer without a fixed price

The term “good deal” implies a price, and a card’s price changes from week to week. Rather than cite one that will be wrong in ten days, here’s the method. Calculate the cost per GB of VRAM: the price divided by 16 for the 7800 XT, or by 12 for a 12 GB card. The site’s tracker publishes this figure for each card. Then ask yourself the only question that matters: does the extra capacity unlock a model you actually want to use? Moving from 12 to 16 GB unlocks 20B to 24B models; moving from 16 to 24 GB unlocks 30B and larger models.

An example of the reasoning, with no pricing implication: if the gap between a 12 GB card and the 7800 XT amounts to what you would have spent anyway on a 24B model, the question is settled. Conversely, if you only use 8B to 12B models, those 4 GB serve no purpose, and the RX 7700 XT or the RX 6700 XT are sufficient.

• Local AI graphics card prices, recorded twice a week
• Which LLM for 16 GB of VRAM, regardless of card

#What fits in 16 GB of VRAM

With 16 GB, allow about 15 GB for the model and its KV cache when the card isn’t displaying the screen. The QuelLLM catalog lists the Q4 weights below; the context is added on top, and headroom quickly becomes the limiting factor on large models.

The key word in the table is headroom: a 24B in Q4 fits, but only with a context of a few thousand tokens. For coding with long files, KV cache quantization frees up one to two gigabytes, and a 20 GB card like the RX 7900 XT solves the problem. The site's table on 16 GB models, independent of the card brand, details the quantization trade-offs.

A word about mixture-of-experts (MoE) models such as gpt-oss 20B: only some of the parameters work on each token, which speeds up generation, but all weights must reside in memory. It is the total size, not the active size, that determines whether the model fits in 16 GB. That explains why a 20B MoE appears alongside a 12B dense model in the table above, with the same weight of about 13 GB.

#Measured throughput: Vulkan or ROCm

The figures below are not measurements from the site: they come from two public discussions in the llama.cpp repository, one for Vulkan and the other for ROCm, both testing Llama 2 7B in Q4_0 with llama-bench. The llama.cpp commits, systems, and drivers differ between runs, so the gap between the two series is an indication, not a definitive ranking.

Two takeaways stand out. First, in these tests, Vulkan generates faster than ROCm on this card (118 versus 101 tokens per second without Flash Attention), while ROCm reads the prompt a little faster. Second, Flash Attention speeds up Vulkan but slightly slows generation under ROCm. If your priority is conversational response speed, try both backends with your model rather than assuming ROCm wins.

Performance relative to the theoretical ceiling helps put things in perspective: 624 GB/s divided by 3.82 GB of weights gives 163 tokens per second; the card reaches 72% of that under Vulkan. Applied to larger models, this ratio provides rough estimates to confirm on your own system.

#Against NVIDIA cards with 16 GB: prompt processing settles the question

The comparison with 16 GB RTX cards comes down to two metrics. For generation, the RX 7800 XT remains in the same range as the NVIDIA cards measured under Vulkan. For prompt processing, the gap is very large: three times slower than a RTX 4070 Ti Super. This second figure matters for use cases involving long documents or long conversation histories, where you're waiting for the first word. It matters little for a short chat or a coding assistant with a modest context.

• RTX 4070 Ti Super, the 16 GB alternative NVIDIA
• Radeon RX 7900 GRE, another 16 GB card from the same generation

#How to fit a 24B model into 16 GB: useful levers

A 24-billion-parameter model in Q4 weighs about 14 GB, leaving barely a gigabyte for the KV cache and system. Four levers prevent overflow onto the processor, which would sharply reduce speed. The first is to limit the context window to what you actually need, because the cache grows with every token. The second is to quantize the KV cache, freeing up one to two gigabytes. The third is to close applications that reserve VRAM, such as a browser with hardware acceleration or a game. The fourth is to connect the display to the processor’s integrated graphics when available, returning to the AMD card the memory used by the display.

Context

Set the window to match your actual needs: 4,000 to 8,000 tokens are enough for most conversations.

KV cache

Quantize it to 8-bit to free up VRAM, while monitoring quality on your tasks.

Display

Use the integrated graphics card’s video output when available.

Verification

After loading, ollama ps should show 100% GPU; any other proportion indicates overflow.

The guide to the context window explains why a long history consumes as much memory as the model itself on smaller cards.

• Understanding the context window
• Quantifying the KV cache: gaining VRAM

#Getting started

1. 01
   Choose the system

   On Linux, AMD supports Radeon RX 7000 only on Ubuntu 24.04.4, Ubuntu 22.04.5, RHEL 10.1, and RHEL 9.7. On Windows, Ollama relies on a ROCm v7 or HIP7 stack.
2. 02
   Install Ollama

   Follow the Ollama documentation: ROCm v7 is required on Linux. Vulkan is enabled by default and serves as a fallback.
3. 03
   Load a model that fits

   Try a 12B in Q8 or a 20B in Q4 before attempting a 24B. Check with ollama ps that it is entirely on the GPU.
4. 04
   Compare Vulkan and ROCm

   Run llama-bench with both backends on your model, with and without Flash Attention, and keep the faster one for your use case.

ollama ps
./llama-bench -m llama-2-7b.Q4_0.gguf -ngl 100 -fa 0,1

• Ollama and AMD GPUs: ROCm, Vulkan, and useful variables

#2026 verdict

A good option if

You’re targeting 20B to 24B models, and the VRAM cost per GB on /prix-gpu-ia is lower than on nearby 12 GB cards.

Not a good choice if

You’re sticking with 8B to 12B models: a 12 GB card is enough, and you’re paying for unused gigabytes.

Avoid

Buying a 12 to 14 GB model without testing whether it loads when the card is used.

• Ollama documentation: supported AMD cards
• ROCm GPU compatibility, AMD documentation
• Vulkan scoreboard for the llama.cpp repository
• ROCm scoreboard for the llama.cpp repository

#Frequently asked questions