How to install local LLMs
Hardware-specific, step-by-step install guides — the exact quantization, tool (Ollama, LM Studio, llama.cpp, vLLM), and settings for the GPU or machine you actually own.
Install DeepSeek R1 32B on RTX 5090
Ollama, LM Studio, or vLLM on the RTX 5090's 32 GB GDDR7 — quantization picks, VRAM math, and setup steps.
Install Llama 3.3 70B on RTX 4090
VRAM math, Ollama vs llama.cpp, and IQ2_XS vs Q4_K_M speed benchmarks for a 70B model on a single RTX 4090.
Install Mistral Small 3.1 24B on RTX 3060
Which GGUF quant fits a 12 GB RTX 3060, Ollama and LM Studio steps, and offload tuning.
Install Ollama on Windows 11 with WSL2
Enable WSL2, install Ubuntu 24.04, and add GPU passthrough for NVIDIA/AMD so Ollama runs at native speed.
Install Qwen 2.5 Coder 32B on Mac M4 Max
Compare LM Studio, Ollama, and llama.cpp on Apple Silicon, and pick the right quant for the M4 Max's unified memory.