Guide · 2026-07-21

How to Wire Continue.dev to a Local Ollama LLM in VS Code

Last updated 2026-07-21

A verdict-driven setup guide: connect the Continue.dev VS Code extension to a local Ollama model for private, subscription-free chat, edit, and autocomplete.

By Mohamed Meguedmi · 9 min read

The verdict: this is the local stack worth wiring

If you want a private coding assistant that never phones home, the pairing of the Continue.dev VS Code extension and the Ollama runtime is the strongest option available in 2026 — and it is the one our editorial team recommends over LM Studio-based or llama.cpp-server setups for one reason: role-based routing. Continue lets you assign a different model to chat, edit, autocomplete, and embeddings in a single file, so you stop trying to force one 32B model to do a job a 1.5B base model does faster and better.

The trap most guides fall into is telling you to point every feature at the same instruct model. That produces a laggy, wrong autocomplete because instruct models are not trained for fill-in-the-middle (FIM). Get the model split right and the experience rivals a cloud assistant. Get it wrong and you will uninstall the extension within an hour. This guide takes a position on exactly which models to run and why.

Hardware you actually need

Ollama runs quantized GGUF weights, so memory footprint — not raw compute — is the gate. The figures below are for Q4_K_M quantization, the sweet spot our benchmarks consistently show for coding tasks. "Unified" refers to Apple Silicon shared memory; discrete-GPU users should read the column as VRAM.

Chat model	Memory (Q4_K_M)	Practical floor	Chat throughput*
Qwen2.5-Coder 7B	~5.0 GB	16 GB RAM / 8 GB VRAM	35–60 tok/s
Qwen2.5-Coder 14B	~9.0 GB	24 GB RAM / 12 GB VRAM	22–40 tok/s
Qwen2.5-Coder 32B	~20 GB	32 GB RAM / 24 GB VRAM	12–25 tok/s

*Measured range across an Apple M3 Pro and an NVIDIA RTX 4070/4090 class GPU. Throughput scales with memory bandwidth, not core count — an M-series chip with high bandwidth often beats a mid-range discrete GPU on these sizes.

The autocomplete model is deliberately tiny (Qwen2.5-Coder 1.5B-base, ~1.0 GB) because latency, not intelligence, is what matters when a suggestion must appear in under 300 ms as you type. Budget roughly 1 GB extra for the embedding model. Want to compare total cost of ownership against a cloud subscription? Run the numbers in our cost calculator.

Step 1 — Install Ollama and pull the right models

Install Ollama for macOS, Windows, or Linux from the official site, then confirm the background service is live. Ollama listens on http://localhost:11434 by default; Continue will talk to it there.

# Verify Ollama is serving
ollama --version
curl http://localhost:11434/api/tags

# Chat + edit model (instruct)
ollama pull qwen2.5-coder:7b

# Autocomplete model (BASE variant — required for FIM)
ollama pull qwen2.5-coder:1.5b-base

# Embeddings for @codebase retrieval
ollama pull nomic-embed-text

The single most common mistake is pulling qwen2.5-coder:1.5b (the instruct build) for autocomplete. You need the -base tag: only the base model carries the FIM special tokens Continue relies on. See the Ollama Qwen2.5-Coder page for the full tag list and the nomic-embed-text page for embedding details.

Step 2 — Install the Continue extension

Open VS Code, go to the Extensions view (Ctrl/Cmd+Shift+X), search Continue, and install the official publisher (Continue.dev). A new Continue icon appears in the activity bar. On first launch it may offer a hosted onboarding flow — skip it. You are wiring a local backend, so you do not need an account or an API key.

Continue stores its configuration in the .continue directory in your home folder: ~/.continue/config.yaml on macOS/Linux, %USERPROFILE%\.continue\config.yaml on Windows. If the file does not exist yet, create it.

Step 3 — Wire them together in config.yaml

This is where the stack lives or dies. Continue moved to a YAML config with explicit roles; the older config.json still loads for backward compatibility but is deprecated. Use YAML. The block below assigns each feature to the model built for it:

name: Local Assistant
version: 1.0.0
schema: v1

models:
  - name: Qwen2.5-Coder 7B
    provider: ollama
    model: qwen2.5-coder:7b
    roles:
      - chat
      - edit
      - apply

  - name: Autocomplete
    provider: ollama
    model: qwen2.5-coder:1.5b-base
    roles:
      - autocomplete

  - name: Embeddings
    provider: ollama
    model: nomic-embed-text
    roles:
      - embed

context:
  - provider: code
  - provider: codebase
  - provider: diff
  - provider: terminal

Save the file. Continue hot-reloads config, so within a second or two you should be able to open the chat panel and ask a question. Start typing in any code file and inline gray-text suggestions should appear — press Tab to accept. Use @codebase in chat to trigger embedding-backed retrieval across your project. Full schema options are documented at the official Continue docs.

If autocomplete is silent, 90% of the time the cause is one of two things: you pulled the instruct model instead of the -base variant, or the autocomplete role is missing from your YAML. Check both before touching anything else.

Which models to run — with numbers

Model choice should be driven by data, not vibes. The table below uses HumanEval pass@1 from the Qwen2.5-Coder technical report to size the chat model against your memory budget. HumanEval is not the whole story, but it is a reliable ranking signal for the single-file completion tasks you will hit most.

Model	Role	HumanEval pass@1	Recommended when
Qwen2.5-Coder 1.5B-base	Autocomplete	43.9%	Always — it is the FIM engine, not the brain
Qwen2.5-Coder 7B	Chat / edit	88.4%	16 GB systems; best balance of speed and quality
Qwen2.5-Coder 14B	Chat / edit	89.6%	24 GB systems wanting a quality bump
Qwen2.5-Coder 32B	Chat / edit	92.7%	32 GB+; near-frontier local coding

Scores per the Qwen2.5-Coder technical report (arXiv:2409.12186); model cards on HuggingFace list per-language breakdowns.

Our recommendation is unambiguous: start with Qwen2.5-Coder 7B for chat and 1.5B-base for autocomplete. It fits the most common hardware, and the jump from 7B to 32B on real refactoring tasks is smaller than the throughput penalty for most day-to-day work. Move up only if you have 32 GB free and routinely feed the model long, multi-file context. Browse alternatives across families in our model catalog, and note that all the benchmark figures behind our comparisons are queryable through the free BestLLMfor public API (CC BY 4.0) and our open-source MCP server if you want to script your own model selection.

Troubleshooting and performance tuning

Autocomplete never fires

Confirm the model tag is qwen2.5-coder:1.5b-base and that an autocomplete role exists in config.yaml. Then check Continue's output log (Output panel → "Continue") for a 404 from Ollama, which means the model is not pulled.

@codebase returns nothing

You are missing the embed role. Pull nomic-embed-text, add the embeddings block above, then re-index via the Continue command palette. Indexing a large repo can take a minute on first run.

Responses are painfully slow

Either the model exceeds your memory and Ollama is spilling to swap, or context length is set too high. Drop to a smaller chat model and keep num_ctx reasonable — 8192 tokens is plenty for most edits and costs far less memory than 32k. Watch ollama ps to confirm the model is loaded on GPU, not CPU.

Continue can't reach Ollama

If Ollama runs on another machine or in a container, set an apiBase under the model in YAML pointing at that host's :11434 endpoint, and make sure Ollama is bound to 0.0.0.0 rather than localhost-only.

Conclusion: the recommended configuration

The winning setup is not the biggest model you can load — it is the right model in each role. Wire chat and edit to a 7B instruct model, autocomplete to a 1.5B base model, and retrieval to a dedicated embedder, and you get a fully private assistant that costs nothing per month and keeps every keystroke on your machine.

Role	Recommended model	Tag	Non-negotiable because
Chat / edit	Qwen2.5-Coder 7B	qwen2.5-coder:7b	88.4% HumanEval, fits 16 GB
Autocomplete	Qwen2.5-Coder 1.5B-base	qwen2.5-coder:1.5b-base	FIM tokens + sub-300ms latency
Embeddings	nomic-embed-text	nomic-embed-text	Without it, @codebase fails

Copy the config.yaml above, pull those three tags, and you are running a Copilot-class local assistant in under ten minutes. Scale the chat model up only when your hardware and workload justify it.