🇨🇳 Qwen 3.8 27B
Qwen 3.8 27B: dense multimodal (text + vision), 262k context, ~16 GB Q4 VRAM (18 GB of Ollama weights). Apache 2.0, agentic coding and vision.
ollama run qwen3.8:27b
Rankings updated on 22/10/2026
Ranking of the LLMs best suited to RAG: long context window (≥ 32k tokens to digest multiple documents), synthesis quality on provided sources, and robustness to “distractors” (irrelevant information in the prompt).
Qwen 3.8 27B: dense multimodal (text + vision), 262k context, ~16 GB Q4 VRAM (18 GB of Ollama weights). Apache 2.0, agentic coding and vision.
ollama run qwen3.8:27b
Cohere 30.5B model focused on agentic coding and reasoning. 488k context, Apache 2.0, ~18 GB VRAM in Q4.
ollama run north-mini-code-1.0
Cloudflare's dense 27B multimodal model, Apache 2.0, 256k context, ~16 GB Q4 VRAM: vision and reasoning on a 16 GB card, official Ollama tag.
ollama pull clef
GLM-4.7-Flash (MoE 31B, ~3B active): the best code/VRAM ratio in the 30B class. MIT, 128k ctx, very fast on 3090/4090.
ollama run glm-4.7-flash
Dense multimodal 27B released April 22, 2026. 262k ctx (1M YaRN). SWE-bench Verified 77.2%.
ollama run qwen3.6:27b
DiffusionGemma 26B (Google): Gemma diffusion-based vision-language model, instruct, 128k context, 15 GB VRAM Q4. Apache 2.0. Released June 2026.
# HuggingFace: google/diffusiongemma-26B-A4B-it
MoE variant of Gemma 4. 26B/4B active. Full multimodal (text+image+audio).
ollama run gemma4:26b
| Rank | Model | Params | Q4 VRAM | Context | License |
|---|---|---|---|---|---|
| #1 | Qwen 3.8 27B | 27B | 16 GB | 262 144 | Apache 2.0 |
| #2 | North Mini Code 1.0 | 30.5B | 18 GB | 488 000 | Apache 2.0 |
| #3 | Clef 27B | 27B | 16 GB | 262 144 | Apache 2.0 |
| #4 | GLM 4.7 Flash | 31B | 19 GB | 128 000 | MIT |
| #5 | Qwen 3.6 27B | 27B | 16 GB | 262 144 | Apache 2.0 |
| #6 | DiffusionGemma 26B-A4B Instruct | 26B | 15 GB | 128 000 | Apache 2.0 |
| #7 | Gemma 4 26B-A4B MoE | 26B | 16 GB | 128 000 | Apache 2.0 |
Your documents, your AI: reliable local RAG for your PDFs, notes, and emails—without sending anything to the cloud.
We keep chat/general models with a context of at least 32k tokens (required for useful chunking). The score favors recent models with large context windows and permissive licenses (enterprise deployment).
Criteria considered:
The scoring is fully transparent: see our methodology for details on VRAM/tokens/sec calculations.
What context size makes for a good RAG?
Minimum 32k tokens to handle 5–10 chunks of ~2k tokens + the prompt. Ideal: 128k tokens (Qwen 3.8 27B has 262,144), so you can take in whole documents without aggressive chunking.
Which model for RAG on 24 GB VRAM (RTX 4090)?
Mistral Small 3.1 24B in Q4_K_M or Qwen 2.5 32B fit in 24 GB in Q4. For Llama 3.3 70B, you need to drop to Q2/Q3 or add a second card.
Which embedding should you pair with these LLMs?
For French: BGE-M3, multilingual-e5-large, or the Mistral embeddings. See the French embeddings guide.
Which local RAG stack is recommended?
Ollama (LLM server) + ChromaDB or Qdrant (vector store) + LlamaIndex or LangChain (orchestration). See the complete guide.