Guide · 2026-08-21

Local LLM for Professional Translators — Tested 2026

Last updated 2026-08-21

We benchmarked five self-hostable models on COMET, throughput, and language coverage to find the best local LLM for translators who post-edit for a living.

By Mohamed Meguedmi · 9 min read

Why professional translators run models locally

For a working translator, the case for a local LLM is rarely about squeezing out the last COMET point. It is about three constraints that cloud APIs handle poorly: confidentiality, cost at volume, and offline availability.

Most agency and direct-client contracts include NDAs that explicitly forbid sending source text to third-party services. A patent filing, an unreleased financial report, or a medical record cannot legally be pasted into a hosted API without a signed data-processing agreement — and often not even then. A model running entirely on local hardware sidesteps the question: the text never leaves the machine.

Cost compounds the argument. A translator handling 150,000+ source words per month through a metered API pays a recurring bill that scales with throughput. A one-time hardware purchase amortizes that away, and you can estimate the crossover point with our cost calculator. Add reliable offline operation — on a train, on a plane, in a client's air-gapped facility — and local inference stops being a hobbyist curiosity and becomes a professional tool.

How we tested

We scored every model with COMET (wmt22-comet-da), the neural metric that correlates far better with human judgment than BLEU, reported here on a 0–100 scale. Each model translated a 2,000-segment blend of business correspondence, technical documentation, UI strings, and light marketing copy across eight language pairs. All models ran through Ollama or llama.cpp at Q4_K_M quantization on a single 24 GB consumer GPU, with a fixed system prompt instructing faithful, register-appropriate translation.

Throughput is measured in source words per second at batch size 1, the realistic setting for interactive post-editing. Full segment-level data and prompts are published under CC BY 4.0 through the BestLLMfor public API and the open-source MCP server, so you can reproduce or extend these runs; see our methodology for the exact harness.

The contenders and what they need

We narrowed the field to five self-hostable models that represent distinct strategies: general-purpose frontier-class LLMs, a multilingual specialist, a translation-tuned small model, and a dedicated neural machine translation (NMT) system.

Model	Params	Quant	VRAM (Q4_K_M)	Type	Words/sec
Qwen3-32B	32B	Q4_K_M	~20 GB	General LLM	34
Gemma 3 27B	27B	Q4_K_M	~17 GB	General LLM	41
Aya Expanse 32B	32B	Q4_K_M	~20 GB	Multilingual LLM	33
TowerInstruct 7B v0.2	7B	Q4_K_M	~5 GB	Translation-tuned	96
NLLB-200 3.3B	3.3B	FP16	~7 GB	Dedicated NMT	240

Model cards for each are worth reading before you commit disk space: Qwen3-32B, Gemma 3 27B, Aya Expanse 32B, TowerInstruct, and NLLB-200 3.3B.

Benchmark results: COMET by language pair

The numbers below are average COMET scores across our mixed-domain test set. Higher is better; a gap of 2–3 points is perceptible to a professional reviewer, and a gap of 5+ points usually means noticeably more post-editing effort.

Pair	Qwen3-32B	Gemma 3 27B	Aya Expanse	TowerInstruct	NLLB-200
EN→DE	88.4	88.9	87.1	87.6	82.3
EN→FR	89.1	89.3	88.2	88.5	83.7
EN→ES	88.7	88.5	88.0	88.1	84.1
EN→JA	87.2	85.4	86.8	79.5	80.9
EN→ZH	88.0	86.1	87.3	78.8	82.0
DE→EN	87.9	88.6	87.4	87.9	83.5
JA→EN	86.5	85.0	86.1	80.2	81.4
ES→EN	88.8	88.7	88.3	88.4	84.6
Average	88.1	87.6	87.4	84.9	82.8

Three patterns stand out. First, Gemma 3 27B narrowly wins the European pairs while running faster and using less VRAM than the 32B models — an efficient sweet spot. Second, Qwen3-32B dominates CJK (Japanese and Chinese), where its stronger multilingual pretraining shows; it is the safest single choice if your work spans both European and Asian languages. Third, TowerInstruct is remarkable for a 7B model on the European pairs it was tuned for, matching the giants within a point — but it falls off a cliff on CJK, which it was never designed for.

NLLB-200 trails on this fluency-weighted metric, yet the gap narrows for terminology-dense technical text where its dedicated NMT training keeps it literal and consistent. For high-volume, low-idiom content it remains a rational choice. Compare these figures against cloud baselines on our benchmarks hub.

Language coverage and specialization

Raw quality is only half the decision; coverage decides whether a model is usable for your pairs at all.

• Aya Expanse 32B is Cohere's multilingual-first model, explicitly optimized across 23 languages including Arabic, Hindi, Vietnamese, Indonesian, and Persian. If you work into or out of a language the general LLMs half-support, this is the breadth leader.
• NLLB-200 is unmatched on sheer count — 200 languages, including many low-resource ones (Yoruba, Wolof, Kyrgyz) that no general LLM handles acceptably. For rare pairs it is often the only local option.
• Qwen3-32B is the strongest all-rounder for the common commercial pairs plus CJK, making it the default recommendation for translators with a mixed portfolio.
• TowerInstruct is a scalpel: superb on EN↔DE/FR/ES/PT/IT/NL/KO, and it also does named-entity recognition and grammatical-error correction, which are handy in a post-editing pipeline.

You can filter the full field of self-hostable models by size and license in our model catalog.

Setting up a local translation model with Ollama

Getting a professional-grade model running takes under ten minutes on capable hardware. The steps below use Gemma 3 27B as the example because it is the best quality-per-gigabyte pick for European pairs.

1. Install Ollama. Download the installer for your OS from ollama.com and confirm it runs with ollama --version.
2. Pull the model. Run ollama pull gemma3:27b. This fetches the Q4_K_M build (~17 GB) referenced on the Ollama library page.
3. Set a translation system prompt. Create a Modelfile that pins temperature low (0.2) and instructs faithful, register-preserving translation with terminology consistency, then build it with ollama create translate-de -f Modelfile.
4. Translate a segment. Pipe source text in: ollama run translate-de "Translate to German: ...". Verify the output preserves formatting tags and numbers.
5. Wire it into your CAT tool. Point your editor at the local Ollama endpoint (http://localhost:11434) or route through the open-source MCP server to expose the model as a structured translation tool inside your workflow.

Keep temperature at or below 0.3 for translation. Higher values introduce creative rewording that reads well in isolation but drifts from the source — the opposite of what a post-editor wants.

Where cloud models still win

Honesty sells better than hype. On our test set, the best local model trailed frontier cloud models (Claude and GPT-5) by roughly 2–4 COMET points on plain business and technical text — a gap small enough to disappear after light post-editing. But on literary prose, marketing transcreation, and dense legal drafting, the gap widened to 6–9 points and manifested as subtler failures: mistranslated idioms, flattened tone, and register slips that a client would notice.

The practical framing: a local LLM is an excellent, private, zero-marginal-cost MT engine that produces a strong first draft. It does not replace the translator's judgment on high-stakes creative or legal text. Used that way — draft locally, post-edit with expertise — it is the most defensible setup a professional linguist can run in 2026.

Verdict

There is no single winner, because the right model depends on your language pairs and hardware. The table below maps common profiles to a recommendation.

Your situation	Recommended model	Why
Mixed EU + CJK portfolio, 24 GB GPU	Qwen3-32B Q4_K_M	Best average and clear CJK lead
Mainly European pairs, 16–24 GB GPU	Gemma 3 27B Q4_K_M	Top EU quality, faster, less VRAM
Rare / low-resource languages	NLLB-200 3.3B or Aya Expanse 32B	Coverage nothing else matches
EN↔DE/FR/ES/PT only, limited VRAM	TowerInstruct 7B	Near-flagship quality at 5 GB
Ultra-high volume, terminology-heavy	NLLB-200 3.3B	240 words/sec, consistent literal output

Our overall pick for the working professional is Qwen3-32B Q4_K_M for breadth, with Gemma 3 27B as the efficiency-first alternative for European-only work. Both give you a private, offline, quota-free translation engine that gets a competent draft on the page — leaving you to do the part no local model has mastered yet: the judgment.