VRAM: what it is and how much you need for AI ?

#VRAM in one minute

VRAM stands for Video Random Access Memory. It is memory, like RAM, but located a few millimeters from the graphics chip and connected to it by a very wide bus. Its original role is to store what the GPU constantly manipulates: the textures, geometry, and images of a game. For AI, it stores the model weights and its working memory.

Two numbers define it. Capacity, in gigabytes, tells you what can fit. Bandwidth, in gigabytes per second, tells you how quickly the GPU can read it. In gaming, capacity is the main concern. In AI, both matter: capacity determines whether the model loads, while bandwidth determines how quickly it writes.

#RAM and VRAM: the differences

The bandwidth gap, on the order of ten to twenty times, explains everything else on this page. An AI model rereads all its weights for every token it produces. From VRAM, this is fast. From RAM, the same model runs five to twenty times slower.

#Know your VRAM

Windows, without installing anything

Open Task Manager (Ctrl + Shift + Esc), select the Performance tab, then GPU in the left column. The “Dedicated GPU memory” line shows your VRAM. The “Shared GPU memory” displayed beside it is not VRAM: it is RAM that Windows can lend to the GPU, and it is much slower.

Windows, alternative method

Windows key + R, type dxdiag, Display tab: the “Display Memory (VRAM)” line.

Windows and Linux with a NVIDIA card

In a terminal, the nvidia-smi command displays total and used memory in real time. It's the tool to keep open while a model is running.

Linux with an AMD card

The rocm-smi --showmeminfo vram command, or your distribution’s graphical tool.

Mac

Apple menu, About This Mac: the Memory line. A Mac with Apple Silicon has no separate VRAM; see the dedicated section below.

nvidia-smi --query-gpu=name,memory.total,memory.used --format=csv -l 2

#Why local AI depends on VRAM

An LLM uses VRAM in three ways. First, the weights: the number of parameters multiplied by the precision. With 4-bit quantization, count about 0.6 GB per billion parameters, or 5 GB for an 8B model. Next, the KV cache, which stores the current conversation and grows with the context length. Finally, an operating margin of around 1 to 2 GB.

If the total exceeds VRAM, tools such as Ollama or llama.cpp offload part of the model to RAM: this is CPU+GPU offloading, a documented llama.cpp feature for running models larger than the available VRAM. The model still works, but speed collapses as soon as a few layers cross over, because each token then has to wait for a read from system RAM, which is ten to twenty times slower. That’s why an older card with lots of VRAM, such as a RTX 3090 with 24 GB, remains more useful for local AI than a recent 12 GB card, even though the latter is faster for gaming and 3D rendering.

• VRAM calculator: weights, context, and headroom for your model
• Choose your quantization: Q4, Q5, Q8, or FP16

#How much VRAM for each model

The table cross-references our database of 90 cards and chips with the 249 models in the catalog. For each tier, it lists a representative model that fits entirely in VRAM in Q4_K_M. Keep 1 to 3 GB of headroom for the context: when the model's size approaches the card's capacity, the conversation must remain short.

• Which LLM for 8 GB of VRAM?
• Which LLM for 12 GB of VRAM?
• Which LLM for 16 GB of VRAM?
• Which LLM for 24 GB of VRAM?

#Can you increase your VRAM?

On a dedicated graphics card, no, regardless of the manufacturer. The memory chips are soldered and sized with the GPU during design: the only way to get more VRAM is to replace the card or add a second one, which local AI tools can use by automatically distributing the model across the cards. Settings that promise to “increase VRAM” in Windows or the BIOS do not create additional memory: they concern two special cases, described below, that provide no benefit to a dedicated card already fully recognized by the system.

Integrated GPU (iGPU)

It has no dedicated VRAM and borrows some system RAM. The BIOS often lets you increase this allocation (the UMA Frame Buffer Size setting or equivalent). This increases capacity, not speed, which remains that of RAM.

Windows shared GPU memory

Windows allows a dedicated card to spill over into RAM. For a game, this prevents a crash. For an LLM, it is the slow scenario described above.

The real room for maneuver is software: choose a lighter quantization, reduce the context window, or quantize the KV cache. These three levers often save several gigabytes without changing the hardware.

• Quantizing the KV cache: save VRAM on long contexts
• Combine the VRAM of two cards with llama.cpp

#The Mac case: unified memory

Macs with Apple Silicon do not have separate VRAM. The CPU and GPU share a single, unified memory pool with high bandwidth. In the M4 generation: 120 GB/s for the base chip, up to 546 GB/s for an M4 Max. The GPU cannot claim all of that memory because macOS reserves part of it; in practice, plan for about 16 GB usable by a model on a 24 GB Mac, 48 GB on a 64 GB Mac, and 96 GB on a 128 GB Mac—a reserve that weighs proportionally more heavily on smaller configurations.

The lineup has changed since then: Apple launched the M5 chip in late 2025 (153.6 GB/s, 32 GB maximum memory), followed by the M5 Pro (307 GB/s, 64 GB) and the M5 Max (up to 614 GB/s, 128 GB). The Mac Studio followed with the M5 Ultra, announced on August 25, 2026: Apple advertises “up to 512 GB” of unified memory and “1.2 TB/s” of bandwidth, theoretically enough to load a model with several hundred billion parameters. That same day, Apple released the M6, built on a 2 nm process and available since September 22, 2026: it is only an entry-level chip, with no Pro or Max variant this generation, capped at 32 GB of memory and 170 GB/s of bandwidth—less than an M5 Pro released a year earlier. One point to check before buying a “latest-generation” Mac for AI: the chipset name does not tell the whole story; the variant matters more than the model year.

This is what makes high-end Macs distinctive for local AI: no consumer graphics card offers 96 GB of usable VRAM, let alone the hundreds of gigabytes in a Mac Studio M5 Ultra. The trade-off is that, at equal capacity, a recent NVIDIA card is still faster thanks to higher bandwidth per gigabyte: the 1,792 GB/s of a RTX 5090 with 32 GB far exceeds the 614 GB/s of an M5 Max with 128 GB, but the latter can load a model four times larger.

• Hardware for local AI: our machine guides
• Which GPU for a local LLM?
• Source: Apple Newsroom, M6 and M5 Ultra (25/08/2026)
• Source: NVIDIA GeForce RTX 5090 specification sheet
• Source: llama.cpp GitHub repository (multi-GPU distribution)

#FAQ

Recommended hardware: GMKtec EVO-X2 64GB / 1TB (Ryzen AI Max+ 395) — 16 GB of VRAM, enough to load a 14B model quantized to Q4 with its context. All AI hardware →