Running Local Models in Claude Code via Ollama
Verdict (September 2026): Claude Code speaks Anthropic's Messages API and Ollama does not, so there is no official "point it at localhost" switch. You bridge the two with a translation proxy that presents an Anthropic-compatible endpoint on top of Ollama. On a 16 GB card you can run capable 7B–14B coders, but expect weaker tool-calling, shorter usable context, and more retries than cloud Claude. It's a genuine workflow for offline, private, or cost-free work — not a drop-in replacement.
Why it isn't a one-line switch
Claude Code is built around Anthropic's Messages API. Ollama is not — it serves an OpenAI-compatible endpoint plus its own native API on localhost:11434. That mismatch is the whole story: there is no supported "set the model to a local one" toggle inside Claude Code, because the request and response shapes don't line up. If you're new to the runtime itself, start with what Ollama is before wiring anything.
Claude Code does read an environment variable for its base URL, so you can redirect it away from Anthropic's servers. The catch: whatever answers at that URL must speak the Anthropic Messages format, tool-calls included. Point it straight at Ollama and you'll get parse errors. The exact env-var names have shifted between releases, so treat the mechanism as stable but confirm the current variable in the official Claude Code docs before you script it.
The bridge: a translation proxy
The reliable pattern is a small proxy that presents an Anthropic-compatible endpoint on one side and calls Ollama on the other. LiteLLM is the most common open-source option; it can expose an Anthropic-format route and map it to an Ollama model. Dedicated routers built specifically for Claude Code exist too, but they change fast, so lean on an actively maintained project and read its current README.
The shape of the setup is stable even if the commands aren't:
- Run Ollama and pull a coding model — check the official library for exact tags.
- Start the proxy, mapping an Anthropic-style model name to your local Ollama model.
- Set Claude Code's base URL (and a placeholder auth token) to the proxy, then launch it as usual.
Because tool-calling is where local models stumble, verify the proxy translates tool-use blocks in both directions before you judge the model itself.
Sizing local coders for 16 GB
The author's RTX 5070 Ti has 16 GB of VRAM, a common ceiling for local coding. Using the standard rules of thumb — Q4_K_M weights ≈ 0.58 GB per billion parameters, plus roughly 20% for KV-cache and overhead at an 8K context — here's what actually fits. For quant background, see quantization explained, and plug your own numbers into the VRAM calculator.
| Model size | Q4_K_M weights | +~20% overhead | Fits 16 GB? |
|---|---|---|---|
| 7B | 4.1 GB | 4.9 GB | Yes, comfortably |
| 8B | 4.6 GB | 5.6 GB | Yes |
| 14B | 8.1 GB | 9.7 GB | Yes — recommended |
| 20B | 11.6 GB | 13.9 GB | Tight, but yes |
| 32B | 18.6 GB | 22.3 GB | No — needs offload |
A 14B coder at Q4_K_M is the sweet spot on 16 GB: enough headroom for a working context window without spilling into system RAM. 32B-class models need offloading or a bigger card, and offloading slaughters the tokens-per-second that an agentic loop depends on.
Expectations vs cloud Claude
Set expectations honestly. Cloud Claude in Claude Code is fast, follows the tool-call protocol precisely, and holds a large context across a long editing session. A local 7B–14B model does none of those as well. The three failure modes you'll hit most:
- Tool-call adherence: agentic coding lives or dies on well-formed tool calls. Smaller models drift from the schema, causing failed edits and retry loops.
- Usable context: you can load a big window, but quality degrades well before the advertised limit, and KV-cache eats VRAM you wanted for weights.
- Throughput: even on a 16 GB GPU, a multi-step task a cloud model finishes in seconds can take minutes locally.
None of this makes it useless — it makes it a different tool. For picking a model, our best local coding models and best Ollama models shortlists are scored on exactly these traits.
When a local backend is worth it
Reach for a local Ollama backend when the constraint is privacy, offline access, or cost rather than raw capability: editing a proprietary codebase with nothing leaving your machine, working on a plane, or running an unlimited number of low-stakes iterations for free. For fast, correct, large-context agentic work, cloud Claude still wins — and it's fine to switch back per task.
If your goal is a fully local coding assistant rather than specifically bending Claude Code, a purpose-built stack may be less friction — see our local copilot kit. Whatever proxy or router you choose, re-test after upgrades: this corner of the ecosystem moves quickly, and today's working config can break on the next release.
Frequently asked questions
Can Claude Code use Ollama models directly?
Not directly. Claude Code talks to Anthropic's Messages API, which Ollama doesn't expose. You point Claude Code's base URL at a translation proxy, such as LiteLLM, that converts Anthropic-format requests into Ollama calls.
What local model works best for Claude Code on 16 GB?
A 14B-class coding model at Q4_K_M leaves room for context on a 16 GB card and gives the best quality-to-fit ratio. 7B–8B models are faster and more reliable for tool-calling but weaker at multi-step reasoning. Test with your own proxy before committing.
Is local Ollama as good as cloud Claude in Claude Code?
No. Expect weaker tool-call adherence, shorter usable context, and more failed edits or retries. It's usable for offline, private, or cost-free work, not a one-for-one replacement.
Do I need a GPU to run Ollama with Claude Code?
No, Ollama runs on CPU, but agentic coding is token-heavy and CPU-only speeds make the loop painfully slow. A GPU with 8–16 GB of VRAM makes the workflow practical.
By Mohamed Meguedmi — independent comparator of locally-runnable LLMs, benchmarked on a real RTX 5070 Ti (data CC BY 4.0). See the local LLM leaderboard and the best Ollama models.