The developer — “Local copilot, 100% offline, zero cloud latency.”
A local coding model wired into your editor. Autocomplete, chat, whole-file refactors — without a single line of your code leaving your machine for Microsoft, OpenAI or Anthropic. Your commit has an API key in it? It stays there.
By the end of this tour, you'll have a fast local inference server running a dedicated coding model, wired into your editor with autocomplete and in-editor chat, plus a terminal-based agent for larger tasks — all fully offline.
Who this tour is for
We'd rather tell you upfront than let you lose half an hour for nothing.
- ✓You code every day and you're tired of pasting snippets into a web UI.
- ✓Your employer restricts GitHub Copilot, ChatGPT or Cursor — for good reasons.
- ✓You work on proprietary code, under NDA, or with secrets in plaintext.
- ✓You have an RTX 4070 Ti+, RTX 4090/5090, or a MacBook Pro M2/M3/M4 with 32 GB+.
- ✓The round-trip to a cloud API breaks your flow.
- ·You have 8 GB of VRAM: doable, but a 7B model is well below Copilot quality for this.
- ·You code an hour a week: a free cloud account will be less hassle.
- ·You want a general chat assistant, not a coding setup: see the beginner tour.
The path in 6 steps
Each step links to a real, detailed guide. The order is deliberate — don't skip steps the first time through.
-
1Step 01 · 5 min
Pick your copilot workflow
Continue.dev (VS Code sidebar autocomplete + chat), Aider (terminal-driven agent), or Cursor with a local model swapped in — compared head to head.
You know which tool actually matches how you work, before installing anything.Read the guide → -
2Step 02 · 3 min
Pick your coding model
The current ranking of local models for autocomplete and chat, filtered for coding strength and VRAM footprint.
You've shortlisted a coding model that fits your GPU.Read the guide → -
3Step 03 · 4 min
Choose your quantization
Q4, Q5 or Q8 — how much quality you actually lose at each level, and which one makes sense for code.
You know exactly which quantized weights to download.Read the guide → -
4Step 04 · 12 min
Build a fast local inference server
Compile llama.cpp from source with GPU acceleration for noticeably more tokens/sec than a default install.
A llama-server binary is running, serving an OpenAI-compatible API on localhost.Read the guide → -
5Step 05 · 8 min
Wire up your editor
Install the Continue extension, point it at your local endpoint, and configure the fill-in-the-middle model for autocomplete.
Tab to accept a suggestion, or open chat on a selection — fully offline.Read the guide → -
+Step bonus · 5 min
Bonus — add an agentic CLI
For bigger tasks — refactoring a module, generating tests — Aider edits files from natural language in the terminal, with git auto-commit.
A terminal agent wired to the same backend, complementary to your editor setup.Read the guide →
The recommended models
Three picks for this tour, lightest to most capable. Click through for the full spec sheet — VRAM, context, benchmarks.
The strongest fully open coding model in this size class. 19 GB at Q4 — fits a single 24 GB GPU with room to spare.
See the page →MIT-licensed, 10 GB at Q4. Strong completion quality without needing a top-tier GPU.
See the page →9 GB at Q4 — comfortable on a 12 GB card, and still noticeably stronger at code than a generic 7B chat model.
See the page →Frequently asked
The real questions readers send in. If yours is missing, reach out via Contribute below.
How does it compare to GitHub Copilot?
On line-by-line autocomplete, a 32B local coder gets very close — sometimes better on recent code patterns. On open-ended chat, cloud models trained specifically for agentic coding keep an edge. You trade some ceiling for full data control.
Will my laptop heat up and drain the battery?
Yes — a GPU running inference bursts to near 100% on every completion. For serious mobile use, stay plugged in. Apple Silicon handles this more gracefully than most discrete GPUs.
Why llama.cpp and not Ollama?
Ollama works fine too, and is the easier starting point. This tour recommends llama.cpp directly for two reasons: faster throughput on large models, and finer KV-cache control, which matters once you're doing heavy autocomplete traffic.
Can I fine-tune on my own codebase?
Yes, but it's a different level of commitment (LoRA plus a capable GPU and a few hours). RAG over your codebase via Continue's context providers is usually enough, and far faster to set up.
What about full agent mode — reading and writing files on its own?
Aider has done this well since 2024, and Continue.dev has an agent mode too. For long autonomous runs, local 14–32B models are still less reliable than the top cloud models — keep a human in the loop rather than running fully unsupervised.
Copilot Kit
Already have the pieces? Skip the trial and error.
The full setup above, packaged — configs, benchmarks and the exact model picks for your GPU.
The confidential pro — “My files never leave my workstation.”
Lawyer, clinician, HR, consultant, accountant — your documents are covered by privilege, NDA, or plain common sense. A local LLM paired with retrieval over your own files…
A question, a typo, a bug?
This tour evolves with every model release. Your feedback is the raw material.