What MCP actually is
MCP — the Model Context Protocol — is an open standard for connecting language models to external tools, files, and data behind one uniform interface. Instead of writing a custom integration for every API a model might touch, you run an MCP server that advertises a set of tools, and any MCP client can discover and call them over a defined wire format. The protocol was published by Anthropic and is maintained as an open spec with SDKs in several languages; the official MCP documentation is the source of truth for the message shapes.
The part that matters for local setups: MCP is vendor and transport neutral. A server that exposes a "search my notes" or "run this query" tool behaves the same whether the model behind it is a hosted frontier model or a 7B running on your own GPU. That decoupling is exactly what makes it worth wiring into a rig running Ollama — you reuse the same tool servers everyone else uses, but the inference stays on your hardware.
Ollama does tools; MCP needs a client
Here is the point that trips people up: Ollama is not an MCP client. Ollama serves the model and implements tool calling — you send a list of tool schemas in the request, the model emits a structured call, your code runs it, and you feed the result back into the conversation. MCP is a separate layer that standardizes where those tools come from and how they execute. Neither knows about the other by default.
So a working stack has three moving parts:
- Ollama — serves the model and returns tool-call objects through its API. See the Ollama repository for the current API surface.
- An MCP host/client — connects to one or more MCP servers, collects their tool definitions, translates them into Ollama tool schemas, then turns the model's tool calls back into MCP
tools/callrequests. - MCP server(s) — the actual tools: a filesystem browser, a database query interface, a web fetcher, and so on.
The client is the glue, and it is where almost all the real work lives. Whatever model you pick must support tool calling in the first place — not every local model does. Check the model card before you commit; our Ollama model list flags which ones expose tool support.
Wiring it up
The exact commands change between releases, so stay generic and confirm tags against the official library. At a high level the flow is always the same: pull a tool-capable model with Ollama, start your MCP servers, point the MCP client at both, and let the loop run. The client keeps calling Ollama and executing tools until the model returns a final answer instead of another tool call.
| Piece | Role | What to check |
|---|---|---|
| Ollama server | Runs the model, returns tool calls | Model card lists tool support |
| MCP client | Bridges Ollama tool calling to MCP | Supports your Ollama API mode |
| MCP servers | Expose the actual tools | Scope and permissions per tool |
| Transport | How client talks to servers | stdio for local, HTTP for remote |
Two practical warnings. First, tool schemas eat context — every server you connect adds its full tool list to the prompt, so a dozen chatty servers can crowd a small model's window. Connect only what the task needs. Second, MCP servers run real code with real permissions; treat an unknown server the way you'd treat any dependency. The official reference server collection is a safe place to start.
Worked example: quelllm-mcp
To make this concrete, our open-source quelllm-mcp server is the example we reach for. It's a small, self-contained MCP server that exposes a focused set of query tools to whatever client connects — the goal is a minimal, readable reference rather than a kitchen sink, so you can see exactly how a tool definition maps to an execution. Because it speaks plain MCP, the same server works unchanged against a hosted model or against a local one behind Ollama.
The pattern it demonstrates is the one you'll reuse everywhere: the server declares each tool with a name, a description, and a typed input schema; the client turns those into Ollama tool schemas; the model picks a tool and fills in arguments; the client runs it and returns structured output. Nothing in that chain is Ollama-specific, which is the whole point of MCP. Tool lists and setup steps evolve, so treat the repository as the authoritative reference and check its README for the current command and tool set rather than copying a snippet that may have aged. If your aim is a coding assistant specifically, the same wiring underpins our local copilot kit, and the best local models for code page covers which models hold up under tool-driven workflows.
Which local models can actually drive tools
Tool calling is more demanding than plain chat: the model has to choose the right tool, produce valid JSON arguments, and decide when to stop. Small models fail at exactly those steps. In practice ~7B is a sensible floor for reliable single-tool use on a 16 GB card, and multi-step tool chains get noticeably steadier at 14B and up. Below that you'll see malformed arguments and tools called at the wrong time.
Sizing uses the standard rules of thumb — Q4_K_M is roughly 0.58 GB per billion parameters, plus about 20% for KV cache and overhead at an 8K context. On the RTX 5070 Ti's 16 GB:
| Model size | Q4_K_M weights | +~20% (8K ctx) | Fits 16 GB? |
|---|---|---|---|
| 7B | ~4.1 GB | ~4.9 GB | Comfortable |
| 8B | ~4.6 GB | ~5.6 GB | Comfortable |
| 14B | ~8.1 GB | ~9.7 GB | Fits |
| 32B | ~18.6 GB | ~22 GB | No |
| 70B | ~40.6 GB | ~49 GB | No |
The trade-off is real: a 14B model leaves plenty of headroom for tool schemas and a longer context, while a 32B won't fit at Q4 without offloading, which kills the latency that makes tool loops usable. Plug your own numbers into the VRAM calculator before you pull anything, and if quantization levels are new to you, our quantization guide explains what you're trading away at each step.
Frequently asked questions
Does Ollama support MCP directly?
Not by itself. Ollama implements tool calling, but it is not an MCP client, so it does not connect to MCP servers on its own. You run a separate MCP client that collects tools from the servers and passes them to Ollama as tool schemas.
What is the difference between tool calling and MCP?
Tool calling is the model feature that lets Ollama emit a structured request to run a function. MCP is the open standard that defines where those tools live and how they are executed. In a full setup, an MCP client translates between the two.
What size model do I need for reliable tool use with Ollama?
About 7B is a practical floor for dependable single-tool use, and 14B or larger is steadier for multi-step tool chains. Below 7B you tend to see malformed arguments and tools invoked at the wrong moment. Always confirm the model card lists tool support.
Is it safe to connect MCP servers to a local model?
MCP servers run real code with real permissions, so treat any server like a dependency you vet before trusting. Connect only the servers a task needs, and prefer the official reference servers or code you can read. Keeping the tool set small also saves context for smaller models.
By Mohamed Meguedmi — independent comparator of locally-runnable LLMs, benchmarked on a real RTX 5070 Ti (data CC BY 4.0). See the local LLM leaderboard and the best Ollama models.