Build Your Local LLM Stack on a Budget — Interactive Picker

Every week a reader asks the BestLLMfor team the same question: what is the cheapest box I can buy today that runs a useful local LLM? The honest answer is “it depends on what you mean by useful,” so we built a structured picker around four budget tiers and three workload profiles — chat, coding, and reasoning. All numbers below come from our own bench harness or vendor model cards published before 2026-05-01. Where we cite throughput, it is sustained tokens per second at 2,048-token context with batch size 1.

How the picker works

Start by fixing two variables: budget and primary workload. Everything else — quantization, runtime, context length — follows. The mistake most buyer guides make is leading with the model. Lead with VRAM, then pick the largest model that fits at Q4_K_M with 8K context headroom, then pick the runtime. The order matters because GPU memory is the only constraint you cannot fix with patience.

Our rule of thumb, validated across roughly 180 community benchmarks aggregated into the BestLLMfor cost calculator: a model in GGUF Q4_K_M occupies (parameters × 0.55) GB of VRAM, plus ~1.5 GB for KV cache at 8K context. A 14B model therefore needs ~9.2 GB, which is why the RTX 3060 12 GB is still the value champion in mid-2026.

Pick your tier

Tier 1 — the $400 entry stack

If your goal is to replace ChatGPT for low-stakes drafting and summarization, you do not need a GPU. A second-hand HP EliteDesk or Beelink SER5 with a Ryzen 7 5700U and 16 GB of DDR4 routinely sells for $380–$450 on eBay. Paired with Qwen3 4B in Q4_K_M, we measured 17.8 tok/s sustained on llama.cpp build 3891 with 6 threads. That is faster than most people read.

Where this tier breaks down is anything code-shaped. Qwen3-Coder 4B exists, but at 4B parameters it hallucinates imports often enough that you will lose more time debugging than you saved. The entry tier is for chat, translation, classification, and embedding generation — full stop.

Tier 2 — the $900 value champion

This is the tier we recommend to 80% of readers. The used RTX 3060 12 GB has been hovering around $200–$240 on the secondary market since Q1 2026, and 12 GB of VRAM lands exactly on the sweet spot for 14B-class models. Pair it with a Ryzen 5 7600, 32 GB of DDR5-6000, a B650 board, and a 650 W PSU and you are at $900 all-in, including a used case and a fresh 1 TB NVMe.

At 42 tok/s, Qwen3-Coder 14B feels indistinguishable from a hosted API for autocomplete inside Continue or Zed. Our internal bench against the LiveCodeBench Python subset shows it within 4 points of Qwen3-Coder 32B at Q4 — a remarkable price/quality ratio.

Tier 3 — the $1,500 performance stack

The used RTX 3090 24 GB remains, in 2026, the single best value GPU for local inference. Prices have stabilized at $650–$780 as the RTX 50-series pulls gamers upmarket. With 24 GB you can serve Qwen3-Coder 32B Q4_K_M with a 16K context, or run DeepSeek-R1-Distill-Qwen-32B with full reasoning traces.

We measured 28 tok/s on Qwen3-Coder 32B Q4_K_M and 24 tok/s on the DeepSeek-R1 distill, both with vLLM 0.7 and flash-attn 2.7. For perspective, that beats a Mac Studio M2 Ultra on the same models by ~15%, at half the price.

If you only buy one component this year, buy a used 3090. It will outlast two consumer GPU generations for inference.

Tier 4 — the silent Mac alternative

The Mac mini M4 Pro 24 GB at $1,999 is the right pick for one specific reader: someone who wants zero noise, low idle power (~6 W vs ~50 W for a 3090 box), and a unified memory model that simplifies multi-model serving. It runs Llama 4 Scout 17B at 31 tok/s under MLX 0.21 and handles 32B models at ~14 tok/s — slower than a 3090, but at one-eighth the power draw.

The catch: you pay a 60% premium per token versus the 3090 build, and you cannot upgrade. For full methodology on how we normalize these comparisons see our benchmarking methodology.

Runtime: Ollama, llama.cpp, or vLLM?

Three runtimes cover 95% of local deployments in 2026. Pick by workflow, not hype.

• Ollama 0.5+ — best for single-user desktop use. Auto-downloads GGUF, exposes an OpenAI-compatible endpoint, integrates with Open WebUI in one command. Slight overhead (~5%) vs raw llama.cpp.
• llama.cpp — best when you need to squeeze every token/s out of CPU or unified memory. Steepest CLI, best portability, ships matrix-tuned kernels for AVX-512 and Apple Silicon.
• vLLM 0.7+ — best for serving multiple users or agents in parallel. PagedAttention scales near-linearly to batch 16 on a 3090, where llama.cpp plateaus at batch 4.

Solo devs should default to Ollama. Teams of 3+ should default to vLLM behind a small FastAPI proxy. We expose all of these recommendations as structured JSON via the BestLLMfor public API (CC BY 4.0), and the open-source MCP server server can plug it straight into any MCP-compatible client.

Quantization: stop overthinking it

Across our bench suite, Q4_K_M loses an average of 1.2 points on MMLU-Pro and 0.8 points on HumanEval+ versus Q8_0 — well within run-to-run variance. Q3_K_M starts to bite (–3.4 MMLU-Pro) and Q2_K is for emergencies only. Unless you are doing safety research, default to Q4_K_M. The exception: reasoning models like DeepSeek-R1 distills are more sensitive; we recommend Q5_K_M when VRAM allows.

Power, noise, and total cost of ownership

Even the heaviest build costs less per year than three months of a $20 SaaS LLM subscription.

Step-by-step: assemble the $900 value stack

1. Source the GPU. Buy a used RTX 3060 12 GB from a reputable seller; budget $220. Verify the original box and run FurMark for 10 minutes before accepting.
2. Build the host. Ryzen 5 7600, MSI B650M Pro, 32 GB DDR5-6000 CL30, 1 TB NVMe, 650 W 80+ Gold PSU, mATX case. Total: ~$680.
3. Install the OS. Ubuntu 24.04 LTS or Fedora 41. Install NVIDIA driver 560+ and CUDA 12.6.
4. Install Ollama. curl -fsSL https://ollama.com/install.sh | sh
5. Pull the model. ollama pull qwen3-coder:14b-q4_k_m — about 8.5 GB.
6. Add a UI. Run Open WebUI in Docker on port 3000 and point it at http://localhost:11434.
7. Verify throughput. ollama run qwen3-coder:14b-q4_k_m --verbose — expect 38–45 tok/s.

Verdict

Bottom line: in 2026 a used RTX 3060 build is the inflection point where local LLMs become objectively cheaper and faster than equivalent SaaS for a single developer. Above that, you are paying for parallelism or silence — both legitimate, but neither necessary to get started.