DeepSeek V4 Pro locally: VRAM, hardware, installation

#DeepSeek V4 Pro at a glance

Output

April 24, 2026 — DeepSeek-AI on HuggingFace, MIT-licensed weights, paper published simultaneously.

Architecture

Dense Mixture-of-Experts: 1.6T total parameters, 49B active per token, 256 experts with top-8 routing.

Warning

Hybrid CSA (Compressed Sparse Attention) + HCA (Hybrid Constrained Attention)—the first implementation at this scale.

Context

1,000,000 native tokens (with extended RoPE and flash attention v3).

Pretraining

32T+ tokens, Muon optimizer (replaces AdamW), mixed FP4+FP8 precision (the most efficient training known).

Thinking modes

3 levels: Non / High / Max. Max mode approaches o4 performance on AIME and GPQA.

License

MIT — commercial use, redistribution, and modification allowed without restriction. More permissive than the Llama license.

#1. CSA+HCA and 1.6T MoE architecture

DeepSeek V4 Pro combines three major innovations, each published separately over the last 12 months and combined here for the first time at trillion scale.

1.6T MoE / 49B active

256 experts per layer, top-8 routing. For each token, only 49 billion parameters compute—hence speed comparable to a dense 50B model despite the total size.

CSA — Compressed Sparse Attention

Compresses the old keys/values in the KV cache using low-rank decomposition. Cuts attention memory by a factor of 4 for contexts >128k.

HCA — Hybrid Constrained Attention

Combines local attention (4k sliding window) and global attention (sparse), depending on the layer. 2× throughput on long contexts vs. dense attention.

mHC — multi-Head Compressor

Cross-head compression mechanism — reduces activation VRAM with no measurable loss in quality.

Muon optimizer

AdamW's successor, converging 1.8× faster during the pretraining of very large models. Adopted by DeepSeek since V3.5.

FP4+FP8 training

FP4 mix on attention layers, FP8 on MLPs. -45% training memory vs. pure FP8, with quality preserved through a proprietary scaling scheme.

#2. The 3 thinking modes (Non / High / Max)

DeepSeek V4 Pro's thinking system is explicit and toggleable, unlike the opaque modes of competing proprietary models.

No mode (default)

Direct responses with no visible chain-of-thought. Minimal latency (~2–5s at 8 tok/s). For chat, translation, and simple text generation.

High mode

Inserts 200-2000 reasoning tokens before the response. +30-40% quality on math, code, and analysis. Triggered via the /think flag in Ollama, or the thinking_mode header in the API.

Max mode

Exhaustive reasoning (up to 16k chain-of-thought tokens). Approaches o4 performance on AIME 2025 (87% vs. 91% for o4). Expensive: 10–30 minutes per request locally.

# Mode High
ollama run deepseek-v4 "Résous : x² + 5x - 14 = 0 /think"

# Mode Max
ollama run deepseek-v4 "Démontre la conjecture de Collatz pour n<100 /think_max"

#3. Hardware required by quantization

#4. Local installation

Three options depending on your infrastructure: llama.cpp for a multi-machine swarm, vLLM for a homogeneous GPU cluster, or MLX-distributed for a Mac Studio swarm.

#Option A — distributed llama.cpp (recommended for multiple machines)

# Sur chaque machine
git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp
make GGML_METAL=1 GGML_RPC=1 GGML_CUDA=0

# Téléchargement modèle (split GGUF, ~960 Go)
huggingface-cli download deepseek-ai/DeepSeek-V4-Pro-GGUF \
  --include "*Q4_K_M*.gguf" --local-dir ./models

# Machine 1 (rpc-server, écoute sur :50052)
./build/bin/rpc-server -H 0.0.0.0 -p 50052

# Machine 2 (rpc-server également)
./build/bin/rpc-server -H 0.0.0.0 -p 50052

# Coordinator (peut être l'une des deux)
./build/bin/llama-cli \
  -m ./models/DeepSeek-V4-Pro-Q4_K_M-00001-of-00020.gguf \
  --rpc 192.168.1.10:50052,192.168.1.11:50052 \
  -ngl 99 -fa -c 32768 \
  -ctk q8_0 -ctv q8_0

#Option B — vLLM on an H100/H200 cluster

pip install vllm>=0.7

vllm serve deepseek-ai/DeepSeek-V4-Pro \
  --tensor-parallel-size 8 \
  --pipeline-parallel-size 1 \
  --max-model-len 1000000 \
  --quantization fp8 \
  --enable-prefix-caching \
  --port 8000

# Endpoint OpenAI-compatible sur :8000
curl http://localhost:8000/v1/chat/completions -d '{...}'

#Option C — Ollama (when the port becomes available)

As of April 25, 2026, Ollama has not yet integrated DeepSeek V4 Pro (community Q4_K_M quants are being released this weekend). Track progress at ollama.com/library.

#5. Benchmarks vs GPT-5, Claude 4.7, Gemini 3

#6. Use the 1M-token context

The 1M context is native (not extended via YaRN), thanks to HCA. It's one of the three or four open-weight models in the world to offer 1M, and the only one to do so with this level of recall quality.

Needle-in-Haystack 1M

98% recovery across the entire context (vs. 76% for Gemini 1.5 Pro at equivalent settings).

Use case 1 — Entire codebases

Linux kernel net/* (~800k tokens) ingested in one go, with cross-file refactoring in a single request.

Use case 2 — Legal research

Civil Code + recent decisions (~600k tokens) in context, with case-law extraction in one pass.

Use case 3 — Accounting analysis

10 years of financial statements + appendices (~400k tokens), with multi-year anomaly detection.

1M KV cache VRAM

With KV Q4 and CSA enabled: ~280 GB. At 1M without compression: ~2 TB (impractical).

#7. Ideal use cases

Scientific research

Math, theoretical physics, computational chemistry. Max mode beats o4 on certain AIME/Putnam benchmarks.

Security / sovereignty

Banks, defense, healthcare, government: 100% self-hosted frontier model, MIT-licensed and auditable. No equivalent other than LLaMA 4 (more restrictive license).

Complex long-horizon agents

The 1M context, Max reasoning, and 49B active parameters make it a better agent than smaller dense models, without the inference cost of GPT-5.

Enterprise-scale RAG

Knowledge base with several million tokens (codebases, internal docs, ISO standards). Direct search without retrieval thanks to the context.

#Limitations and alternatives

Prohibitive hardware for individuals

Minimum 960 GB VRAM for Q4 = an approximately €80k used cluster. Out of reach for an individual.

Tooling immature at J+1

Community GGUF quants are in progress, vLLM requires the dev branch, and MLX-distributed is still experimental. Wait ~2 weeks for stability.

Carbon footprint

8× H200 cluster 24/7 = ~45 MWh/year. Weigh that against API usage (which amortizes costs better).

More accessible alternatives

DeepSeek V4 Flash 284B (170 GB Q4, fits Mac Studio) · DeepSeek V3.2 671B (240 GB Q2 on Mac Studio 512) · Llama 4 Behemoth (close in quality).

#FAQ