Guide · 2026-07-23

How to Add a Local LLM to Claude Desktop via MCP

Last updated 2026-07-23

Bridge Ollama or LM Studio into Claude Desktop with a lightweight MCP server, so a private local model handles sensitive work while Claude orchestrates.

By Mohamed Meguedmi · 8 min read

What "adding a local LLM" to Claude Desktop actually means

There is a common misconception worth clearing up first: Claude Desktop is a client for Anthropic's cloud models, and the Model Context Protocol (MCP) does not let you swap the underlying Claude model for a local one. What it lets you do is far more useful in practice — it gives Claude a standardized way to call external tools and data sources. A local LLM becomes one of those tools.

MCP is an open protocol maintained by Anthropic that standardizes how LLM applications talk to tools and data (modelcontextprotocol.io). By running a small bridge MCP server, you expose an endpoint like ask_local_model that Claude can invoke. Claude decides when to delegate; your local model — served by Ollama or LM Studio — does the private inference and returns text. Sensitive files, proprietary code, or air-gapped data never leave the machine for that sub-task.

The pattern that wins is a hybrid one: Claude handles reasoning and orchestration; a local model handles the parts you can't or won't send to the cloud.

If you're still deciding which local model to serve, the model catalog and benchmarks pages are the fastest way to size expectations against your hardware.

Prerequisites and hardware

You need three things: Claude Desktop (macOS or Windows), a local inference server, and Node.js 18+ (most bridge servers ship as an npx package). Below is a realistic VRAM and throughput table, measured on a single RTX 4090 (24 GB) at 4-bit quantization.

Model	Quant	Disk	VRAM used	~Tokens/sec	Good for
Llama 3.1 8B	Q4_K_M	4.9 GB	~6 GB	95–110	General chat, summarizing
Qwen3 14B	Q4_K_M	9.0 GB	~11 GB	55–65	Balanced reasoning
Mistral Small 24B	Q4_K_M	14.3 GB	~16 GB	38–45	Document analysis
Qwen3-Coder 32B	Q4_K_M	19.8 GB	~22 GB	28–34	Private code review

On CPU-only or Apple Silicon without enough unified memory, everything still works, but throughput drops to roughly 5–15 tokens/sec for an 8B model — usable for background tasks, painful for interactive ones. Estimate the electricity and amortized cost of running these models locally versus API calls with the cost calculator.

Step-by-step: bridge Ollama into Claude Desktop

This is the reference path. Ollama gives you a stable local API on port 11434, and the bridge server does the MCP translation.

1. Install Ollama and pull a model. Download from ollama.com/download, then pull a model. For a general-purpose start:

   ollama pull llama3.1:8b
   ollama run llama3.1:8b "hello"   # confirms the server is live

   Ollama now serves an OpenAI-compatible endpoint at http://127.0.0.1:11434.
2. Locate the Claude Desktop config file. Open Claude Desktop → Settings → Developer → Edit Config, or edit it directly:
   • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
   • Windows: %APPDATA%\Claude\claude_desktop_config.json
3. Register the bridge MCP server. Add an mcpServers block. This uses the open-source BestLLMfor bridge, which auto-discovers running Ollama models and exposes them as MCP tools:

   {
     "mcpServers": {
       "local-llm": {
         "command": "npx",
         "args": ["-y", "@bestllmfor/local-llm-mcp"],
         "env": {
           "OLLAMA_HOST": "http://127.0.0.1:11434",
           "DEFAULT_MODEL": "llama3.1:8b"
         }
       }
     }
   }

4. Restart Claude Desktop completely. Quit from the tray/menubar, not just the window — MCP servers are spawned only on a full restart.
5. Verify the tool loaded. Open a new chat and look for the tools icon (a small slider/hammer). You should see ask_local_model listed. Ask Claude: "Use the local model tool to summarize this text privately." If Claude calls the tool and returns a response, the bridge is working.

Prefer a one-click install? Claude Desktop now supports desktop extensions — bundled .mcpb files that carry the server and its dependencies, so you skip the JSON entirely (Anthropic support). Double-click the .mcpb, approve it, done. We still recommend the manual route the first time — when something breaks, you'll know exactly which layer to inspect.

Ollama vs LM Studio as the backend

Both work. The choice comes down to whether you want a headless CLI daemon or a GUI with a model browser.

Criterion	Ollama	LM Studio
Interface	CLI + background daemon	Full GUI
Default API port	11434	1234
Model management	ollama pull	In-app search + download
Auto-start on boot	Yes (service)	Manual / GUI
Best for	Reproducible, scriptable setups	Beginners, model experimentation
Bridge maturity	Widely supported	Widely supported

For a config you can commit to version control and reproduce on another machine, Ollama wins. LM Studio is the better on-ramp if you want to try five models before committing. Either way, point the bridge's env at the right host and port.

Choosing what the local model should do

Don't ask an 8B local model to out-reason Claude — it won't, and that's fine. Assign it the tasks where locality is the value, not raw capability:

• Privacy-scoped work: redacting PII, scanning a codebase for secrets, or summarizing an NDA-covered document before anything reaches the cloud.
• Bulk, cheap operations: classifying hundreds of log lines where per-token API cost would add up.
• Offline resilience: keeping a fallback that answers when the network is down.

For private code review specifically, a 4-bit Qwen3-Coder 32B is the sweet spot if you have ~22 GB of VRAM; drop to Qwen3 14B on smaller cards. Model cards on Hugging Face list context windows and license terms — check them before deploying in a business setting. Our own methodology page documents how these throughput and quality figures are produced.

Troubleshooting the three failures you'll actually hit

• Tool doesn't appear. Almost always a JSON syntax error (trailing comma) or a partial restart. Validate the config, then fully quit Claude Desktop. Check the logs at ~/Library/Logs/Claude/mcp*.log (macOS) or %APPDATA%\Claude\logs\ (Windows).
• "Connection refused" from the bridge. The inference server isn't running or the port is wrong. Confirm with curl http://127.0.0.1:11434/api/tags for Ollama. Match OLLAMA_HOST exactly.
• Model loads but responses are slow or truncate. You're over VRAM budget and spilling to system RAM, or the context window is too small. Drop a quant level (Q4 → Q3) or a size tier, and raise num_ctx only as far as memory allows.

Want programmatic access to the same model/hardware data we cite here? The BestLLMfor public API exposes it under a CC BY 4.0 license, and the bridge server referenced above is open source — fork it, add custom tools, self-host with zero telemetry.

Verdict

Adding a local LLM to Claude Desktop is not about running Claude offline — it's about giving Claude a private, local delegate. The most reliable, reproducible setup in 2026 is Ollama plus an open-source bridge MCP server, registered through claude_desktop_config.json. Use the one-click .mcpb extension once you understand the moving parts.

Your situation	Recommended setup	Why
Reproducible, versioned config	Ollama + BestLLMfor bridge (manual JSON)	Scriptable, transparent, easy to debug
Non-technical / fastest start	LM Studio + .mcpb extension	GUI model browser, one-click install
Private code review, 24 GB GPU	Ollama + Qwen3-Coder 32B Q4_K_M	Best local coding quality that fits VRAM
8 GB GPU or laptop	Ollama + Llama 3.1 8B Q4_K_M	90+ tok/s, comfortable memory headroom
CPU-only / air-gapped	Ollama + 8B model, background tasks only	Works, but expect 5–15 tok/s

Compare local models head-to-head on the best-of rankings before you commit a download.