Guide · 2026-07-11

How to Install DeepSeek R1 32B on RTX 5090

Last updated 2026-07-11

The RTX 5090's 32 GB of GDDR7 runs the DeepSeek R1 32B distill entirely in VRAM. Here is the fastest, cleanest way to get it serving locally.

By Mohamed Meguedmi · 9 min read

Why the RTX 5090 is the natural home for a 32B model

The RTX 5090 shipped in January 2025 with 32 GB of GDDR7, 1,792 GB/s of memory bandwidth, 21,760 CUDA cores, and a 575 W board power. For local inference, bandwidth is the number that matters most: token generation is memory-bound, so decode speed scales almost linearly with how fast the GPU can stream weights. The 5090's bandwidth is roughly 78% higher than the RTX 4090's 1,008 GB/s, and that gap is exactly what you feel when generating long reasoning chains.

DeepSeek's 32B release is DeepSeek-R1-Distill-Qwen-32B — a Qwen-2.5 32B base distilled on reasoning traces from the full 671B R1 model. At 4-bit it lands around 20 GB, which is the first time a single consumer card can hold a genuinely strong 32B reasoner plus a usable context window without spilling layers to system RAM. On a 24 GB card (3090/4090) you can run the same weights, but context and KV cache eat into headroom fast; on the 5090 you simply stop worrying about it.

Step 0 — Prepare the driver and CUDA toolkit

The 5090 uses NVIDIA's Blackwell architecture with compute capability sm_120. Any inference stack built before Blackwell support landed will either refuse to compile kernels or quietly run on CPU. Before installing anything model-related:

1. Install NVIDIA driver 570 or newer. Confirm with nvidia-smi — it should report the RTX 5090 and 32 GB of memory.
2. Install the CUDA 12.8+ toolkit if you plan to use vLLM or build anything from source. Ollama and LM Studio ship their own runtimes, so this is optional for those two paths.
3. Reboot after driver changes so the kernel module reloads cleanly.

If nvidia-smi shows the card but your inference tool reports 0 GB of GPU memory or falls back to CPU, you are almost always on a pre-Blackwell CUDA build. Update the tool, not the driver.

Choosing your quantization

Quantization is the single biggest lever on the 5090. Because you are not VRAM-constrained the way smaller cards are, you get to trade a little speed for quality. The table below assumes an 8K context window; larger windows add KV cache on top (roughly 1–2 GB per additional 8K at these sizes).

Quant	Weights on disk	VRAM @ 8K ctx	Fits 5090?	Verdict
Q4_K_M	~19.9 GB	~22–23 GB	Yes, with big headroom	Recommended default. Best speed/quality balance.
Q5_K_M	~23.0 GB	~25–26 GB	Yes	Slightly better fidelity; ~10% slower decode.
Q6_K	~27.0 GB	~29–30 GB	Yes, tight	Marginal quality gain; leaves little room for long context.
Q8_0	~34.8 GB	>32 GB	No (offloads)	Requires CPU offload — avoid on a single 5090.

The takeaway: Q4_K_M is the right starting point for nearly everyone. Step up to Q5_K_M only if you have a specific fidelity-sensitive task and can accept the speed hit. Q8_0 does not fit and defeats the purpose of a single-card build.

Path A — Ollama (fastest install)

Ollama is the shortest route from zero to a running model. It bundles a Blackwell-ready llama.cpp runtime and pulls a pre-quantized Q4_K_M by default.

1. Install Ollama from ollama.com (Linux: curl -fsSL https://ollama.com/install.sh | sh; Windows/macOS: the desktop installer).
2. Pull and run the model in one command:

   ollama run deepseek-r1:32b

3. Wait for the ~20 GB download, then start chatting. The deepseek-r1:32b tag maps to the Qwen-distilled 32B at Q4_K_M.

To confirm the model is fully on the GPU, run ollama ps — the PROCESSOR column should read 100% GPU. To widen the context window, set it explicitly:

OLLAMA_CONTEXT_LENGTH=32768 ollama serve

Ollama exposes an OpenAI-compatible endpoint on http://localhost:11434/v1, so you can point existing clients or agent frameworks straight at it.

Path B — LM Studio (GUI and model management)

If you prefer a graphical tool with a model browser, side-by-side quant comparison, and a built-in chat UI, LM Studio is the cleanest option and ships a CUDA runtime that already targets Blackwell.

1. Install LM Studio and open the Discover tab.
2. Search DeepSeek R1 Distill Qwen 32B and download the Q4_K_M GGUF.
3. In the load dialog, set GPU offload to max (all layers) and context to 16384 or 32768.
4. Load the model and confirm the resource meter shows the full model in VRAM.

LM Studio can also run a local server from the Developer tab, giving you the same OpenAI-compatible API as Ollama without touching a terminal.

Path C — vLLM (throughput and serving)

For batched serving, multiple concurrent users, or agent workloads that fire many requests, vLLM extracts far more aggregate throughput than llama.cpp-based tools thanks to continuous batching and PagedAttention. Use an AWQ or GPTQ 4-bit checkpoint rather than GGUF here.

pip install vllm
vllm serve casperhansen/deepseek-r1-distill-qwen-32b-awq \
  --quantization awq \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.92

Cap --gpu-memory-utilization at ~0.90–0.92 so the KV cache and CUDA graphs have breathing room. Single-stream decode with vLLM is comparable to Ollama, but throughput under concurrency is where it pulls ahead — expect 3–5x the aggregate tokens/sec at batch sizes of 8–16.

Measured performance on the RTX 5090

The numbers below are single-stream figures at an 8K context window, DeepSeek-R1-Distill-Qwen-32B. Treat them as a realistic band rather than a guarantee — thermals, driver version, and prompt length all move the needle. The RTX 4090 column is included for reference at the same Q4_K_M quant.

Tool	Quant	RTX 5090 decode	RTX 4090 decode	Prompt (prefill)
Ollama	Q4_K_M	~48–52 tok/s	~34–37 tok/s	~1,900 tok/s
LM Studio	Q4_K_M	~47–51 tok/s	~33–36 tok/s	~1,850 tok/s
Ollama	Q5_K_M	~43–46 tok/s	~30–32 tok/s	~1,800 tok/s
vLLM (AWQ, batch 1)	4-bit	~50–55 tok/s	~36–39 tok/s	~2,600 tok/s
vLLM (AWQ, batch 8)	4-bit	~230–260 tok/s agg.	~160–180 tok/s agg.	—

A ~50 tok/s decode is comfortably faster than reading speed, so interactive use feels instant even when the model emits a long <think> block before answering. Reasoning models spend a lot of tokens thinking, which is precisely why the 5090's bandwidth advantage compounds: the faster you stream those hidden tokens, the sooner you see the final answer. For a broader cross-model view, see our benchmarks hub, and to model electricity and amortized hardware cost use the cost calculator.

Optimization tips that actually move the needle

• Right-size context. Do not set 128K "because you can." A larger KV cache consumes VRAM and slows prefill. 16K–32K covers almost every reasoning and coding task.
• Enable flash attention where the tool exposes it — it reduces KV memory and speeds long-context prefill. It is on by default in vLLM and modern Ollama builds.
• Watch the power/thermal envelope. At 575 W the 5090 runs hot; sustained decode will throttle in a poorly ventilated case. A modest power limit (e.g. 500 W) often costs only a few percent of speed while cutting temps.
• Set the temperature the model expects. DeepSeek recommends ~0.6 for R1-style reasoning; higher values degrade the chain-of-thought quality.
• Keep the thinking tokens. Do not strip the <think> section from the prompt template — the distill's accuracy depends on it.

Where this fits in the local-LLM landscape

The 32B distill is the practical ceiling for a single 5090 if you want quality context. The 70B distill also runs at 4-bit on this card but leaves far less room for context and drops decode into the ~18–24 tok/s range — usable, but a different experience. If your workload is coding-heavy, it is worth A/B testing the DeepSeek distill against a same-size Qwen coding model on your own prompts; browse comparable options in our model catalog. Every benchmark on this site is reproducible and the underlying data is available through the BestLLMfor public API (CC BY 4.0) and our open-source MCP server, so you can pull these figures directly into your own tooling rather than screenshotting a table.

Verdict

The RTX 5090 is the first consumer card that makes running a 32B reasoning model feel effortless — enough VRAM to hold Q4_K_M weights and a real context window, and enough bandwidth to keep decode above reading speed even through long reasoning chains. Install driver 570+, pull deepseek-r1:32b, and you are running in minutes. Match the path to your goal:

Use case	Recommended path	Quant	Expected decode
Fastest setup / daily use	Ollama	Q4_K_M	~48–52 tok/s
GUI + easy quant swapping	LM Studio	Q4_K_M	~47–51 tok/s
Highest fidelity, single user	Ollama / LM Studio	Q5_K_M	~43–46 tok/s
Concurrent serving / agents	vLLM (AWQ)	4-bit	~230–260 tok/s agg. (batch 8)

For most readers the answer is simple: Ollama at Q4_K_M. It is the shortest path to a fast, fully-in-VRAM DeepSeek R1 32B on the RTX 5090, with room to grow into vLLM once you need to serve more than yourself. For more single-card setup walkthroughs, see our guides hub.