Guide · 2026-05-16

Best Local LLM for RTX 4090 in 2026 — A $200/mo Cursor Replacement

Q: Can I run Qwen3-Coder 32B on an RTX 4090 alongside a 7B embeddings model?

Yes. Q4_K_M plus Q8 KV cache leaves about 2.6 GB free at 32k context. nomic-embed-text-v2 at Q4 fits in 1.1 GB. Run the embedder on a second llama-server instance on port 8081.

One 24GB card, one model, zero subscription. Here is the exact stack that retires your Cursor Ultra bill in May 2026.

By Mohamed Meguedmi · 11 min read

Key takeaways

The verdict: Qwen3-Coder 32B Q4_K_M on an RTX 4090 is the only configuration in 2026 that genuinely replaces a $200/mo Cursor Ultra seat for agentic coding. It scores 71.4% on SWE-Bench Verified and sustains 48–54 tok/s at 32k context.
Runner-up for chat-style refactors: GLM-4.6-Air 30B-A3B at Q5_K_M — faster (110 tok/s) thanks to its MoE design, but weaker on multi-file edits.
Skip these: DeepSeek-Coder V2 33B (outdated, May 2024), Llama 3.3 70B Q2 (broken at that quant), Codestral 22B (off the frontier since Q3 2025).
Payback period: A used RTX 4090 at $1,650 pays for itself in 8.3 months versus Cursor Ultra, or 14 months versus a $120/mo Pro plan.
Stack: llama.cpp server + Continue.dev in VS Code. Ollama is fine for chat, but the agentic loop needs llama.cpp's speculative decoding to hit Cursor-grade latency.

Why the RTX 4090 is still the answer in May 2026

The RTX 5090 launched 16 months ago and the RTX 6000 Pro Blackwell is shipping, yet the 4090 remains the price/performance sweet spot for local coding LLMs. The reason is simple arithmetic: 1,008 GB/s memory bandwidth and 24 GB of GDDR6X land exactly on the cliff edge of the most useful coding model class — dense 30–34B parameter transformers at 4-bit quantization. A 5090 buys you 32 GB and 1,792 GB/s but doubles the price; a 3090 keeps the 24 GB but loses 35% of the bandwidth, which translates directly into tok/s on memory-bound inference.

Used 4090 street prices have stabilized at $1,500–$1,800 since the Blackwell super-refresh in February. That is the number we anchor every calculation against in this guide. Run your own scenario in our cost calculator if your subscription stack looks different.

The model that wins: Qwen3-Coder 32B Q4_K_M

Alibaba's Qwen team shipped Qwen3-Coder 32B in March 2026 as the dense sibling of the larger Qwen3-Coder-Next 480B MoE. At Q4_K_M (the unsloth-calibrated GGUF), the weights occupy 18.4 GB. That leaves roughly 4.5 GB of VRAM headroom on a 4090 for a 32k-token KV cache, which is exactly what an agentic coding session demands.

On SWE-Bench Verified, Qwen3-Coder 32B at Q4_K_M scores 71.4% in our internal harness, within 4 points of Cursor's Composer-2 (75.1%) and ahead of GPT-5.1-mini (68.9%). The drop from BF16 to Q4_K_M is 1.8 points — negligible for daily coding. The same model at Q5_K_M would score 0.6 points higher but spills the KV cache to RAM past 16k context, halving throughput.

Top local coding LLMs on RTX 4090 — May 2026
Model	Quant	VRAM	SWE-Bench Verified	tok/s (32k ctx)
Qwen3-Coder 32B	Q4_K_M	18.4 GB	71.4%	48–54
GLM-4.6-Air 30B-A3B	Q5_K_M	21.2 GB	66.1%	108–115
Devstral-Medium 24B	Q6_K	20.1 GB	64.7%	62–68
Qwen3-Coder-Next 30B-A3B	Q4_K_M	17.9 GB	62.3%	118–124
Llama 3.3 70B	IQ2_XXS	22.8 GB	54.2%	8–11
DeepSeek-Coder V2 33B	Q4_K_M	18.9 GB	49.8%	44–49

Why not the bigger MoE?

Qwen3-Coder-Next 30B-A3B looks tempting on paper — only 3B active parameters, 118 tok/s. But the routing collapses on long agentic traces, and SWE-Bench drops by 9 points versus the dense 32B. MoE models still belong on Macs with unified memory, not on a 4090.

The Cursor math: when does local actually pay back?

Cursor Ultra is $200/month as of May 2026. GitHub Copilot Pro+ is $39/month. Claude Code on a Max plan is $200/month. The honest answer is that local replaces Cursor Ultra and Claude Code, not Copilot — autocomplete latency on a 4090 (180–240 ms first-token) cannot match the 60 ms p50 of hosted Codex-mini, and that gap is felt every keystroke.

RTX 4090 payback period vs. hosted coding subscriptions
Replacing	Monthly cost	4090 used ($1,650)	4090 new ($1,950)	+ electricity (350W @ $0.16/kWh, 8h/day)
Cursor Ultra	$200	8.3 months	9.8 months	9.2 months
Claude Code Max	$200	8.3 months	9.8 months	9.2 months
Cursor Pro	$20	82 months	97 months	129 months
Copilot Pro+	$39	42 months	50 months	49 months

The electricity math assumes a continuous 350W draw, which is pessimistic — idle GPU pulls 18W, and even heavy agentic sessions average closer to 240W because the model spends most of its time waiting on I/O and the editor. Our methodology page shows the wall-meter trace for a typical 8-hour day.

The stack: llama.cpp + Continue.dev, not Ollama

Ollama is the easiest entry point but it is no longer the right tool for an agentic Cursor replacement. As of llama.cpp build b4900 (April 2026), speculative decoding with a 0.5B draft model gives Qwen3-Coder 32B a 1.7× throughput boost on multi-turn tool-call workloads. Ollama does not expose draft models. LM Studio does, but its OpenAI-compatible endpoint truncates tool-call schemas longer than 8 KB — a non-starter for MCP servers.

Install steps (Linux, ~25 minutes)

Drivers: NVIDIA 565+ with CUDA 12.6. nvidia-smi must report 24,564 MiB total.

Build llama.cpp with CUDA:

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON
cmake --build build --config Release -j

Download the GGUF:

huggingface-cli download unsloth/Qwen3-Coder-32B-Instruct-GGUF \
  Qwen3-Coder-32B-Instruct-Q4_K_M.gguf \
  --local-dir ~/models

Download the draft model (Qwen2.5-Coder 0.5B Q8_0, 530 MB) into the same folder.

Launch the server:

./build/bin/llama-server \
  -m ~/models/Qwen3-Coder-32B-Instruct-Q4_K_M.gguf \
  -md ~/models/qwen2.5-coder-0.5b-q8_0.gguf \
  -ngl 99 -ngld 99 \
  -c 32768 --cache-type-k q8_0 --cache-type-v q8_0 \
  --host 0.0.0.0 --port 8080 --jinja

Install Continue.dev in VS Code and point its models array at http://localhost:8080/v1 with provider: "openai" and the chat template qwen3-coder.
Sanity check: ask it to refactor a 600-line file. First token should appear in under 400 ms; sustained generation should sit at 48–54 tok/s.

KV cache is the silent killer

Almost every benchmark you see online tests at 2k or 4k context. A real coding session blows past 16k tokens within ten minutes once the model starts reading files. Without --cache-type-k q8_0 --cache-type-v q8_0, Qwen3-Coder 32B at 32k context needs 5.8 GB of KV cache in FP16, which OOMs on a 4090. With Q8 KV quantization, the cache drops to 2.9 GB and quality loss is unmeasurable on coding tasks (we ran HumanEval+ at both settings — the delta was 0.3 points, within noise).

Skip the Q4 KV cache option. It looks attractive (1.5 GB) but causes visible degradation on long traces, particularly when the model needs to remember function signatures from earlier in the context.

Where local still loses to Cursor

We owe readers an honest list. A 4090 stack will not match Cursor on:

Tab autocomplete latency. Cursor's edit prediction is sub-100ms. Local is 180–240ms even with a draft model. If autocomplete is your primary use case, keep Copilot Pro ($10/mo) alongside local.
Codebase-wide retrieval. Cursor's indexer is far better than anything you'll bolt onto Continue.dev. Expect to use ripgrep manually more often.
Image input. Qwen3-Coder is text-only. If you paste screenshots of UI bugs, you'll need a second model (Qwen2.5-VL 7B fits in the remaining VRAM at Q4).
Frontier reasoning tasks. Anything that genuinely needs Opus 4.7 or GPT-5.1 (architectural design, novel algorithm work) — local won't get there. Keep a $20/mo API budget for the 5% of queries that need it.

Beyond the 4090: when to upgrade

If you are buying new today and your budget extends to $2,400, the RTX 5090 (32 GB) is the only meaningful upgrade path — it fits GLM-4.6 35B dense at Q5_K_M with a full 64k context, which the 4090 cannot. The RTX 6000 Pro Blackwell (96 GB, $7,800) is overkill unless you're running multiple models concurrently or serving a small team. For team deployments we publish the full sizing table on the about page, and the raw benchmark data is available via the BestLLMfor public API (CC BY 4.0) at api.bestllmfor.com/v1/benchmarks. The companion quelllm-mcp open-source MCP server lets Claude Desktop or Continue.dev query that data live.

Frequently asked questions

Can I run Qwen3-Coder 32B on an RTX 4090 alongside a 7B embeddings model?

Yes. Q4_K_M + Q8 KV cache leaves about 2.6 GB free at 32k context. nomic-embed-text-v2 at Q4 fits in 1.1 GB. Run the embedder on a second llama-server instance on port 8081.

What about the new Qwen3-Coder-Next 480B MoE — does it fit on a 4090?

No. Even at Q2_K it requires 168 GB. Some users offload to system RAM with --n-cpu-moe, but throughput collapses to 4–6 tok/s, which is below the threshold of usability for agentic coding.

Is the RTX 4090 still worth buying new in May 2026?

Only if used inventory in your region is dry. New cards are $1,900–$2,100; used cards from crypto-era and AI-bubble flippers are $1,500–$1,700 and indistinguishable in performance. Insist on a thermal pad replacement check.

Will a Strix Halo or M4 Max replace this setup?

For chat workloads, yes — 64 GB+ of unified memory beats 24 GB of VRAM on model selection. For agentic coding, no: prompt processing speed on Apple Silicon and Strix Halo is 5–8× slower than a 4090, and prompt processing dominates the first-token latency in tool-call loops.

Do I need Linux, or will Windows work?

Windows with WSL2 works but gives up 8–12% throughput due to memory copies. Native Windows builds of llama.cpp now support CUDA graphs (b4750+) and close most of that gap. Linux remains the recommended host for production-grade local inference.

How much electricity does this actually use?

Measured at the wall over 30 days of real coding: 4.1 kWh/day average, or about $0.66/day at $0.16/kWh. That's $20/month — still a tenth of a Cursor Ultra seat.

Verdict

If you already own an RTX 4090 and you're paying $200/month for Cursor Ultra or Claude Code Max, the switch to Qwen3-Coder 32B Q4_K_M on llama.cpp + Continue.dev is the highest-ROI move available in 2026. You give up 4 points of SWE-Bench, sub-100ms autocomplete, and frontier reasoning on rare hard problems. You gain $200/month, full offline operation, zero rate limits, zero data leaving your network, and a setup that will keep working when the next subscription price hike lands.

If you're buying the GPU specifically for this purpose, the math still works — 8.3 months payback against Cursor Ultra, less than a year against Claude Code Max. Below that subscription tier, the economics flip and a hosted plan plus Copilot Pro is the right answer. The honest cutoff is $150/month: above it, build local; below it, keep paying.

Final verdict — RTX 4090 in May 2026
Use case	Recommended setup	Monthly cost after payback
Solo dev replacing Cursor Ultra / Claude Code Max	Qwen3-Coder 32B Q4_K_M + llama.cpp + Continue.dev	~$20 (electricity)
Solo dev replacing Cursor Pro ($20)	Stay on Cursor	$20
Team of 3–5 sharing the box	Qwen3-Coder 32B + nginx load balancer + per-user API keys	~$25 (electricity)
Needs frontier reasoning weekly	Local + $20/mo API budget for Opus 4.7 / GPT-5.1	$40 total