Mistral Small 3

Overview

Mistral AI's 24B dense model that closes most of the gap with 70B-class models. Best quality-per-parameter we've measured at this size in 2025.

When to pick this model

• Self-hosting a near-frontier assistant on a single 24GB GPU
• Agentic workflows and tool calling where latency matters
• Long-context RAG with up to 128k tokens
• Commercial deployments needing Apache 2.0
• Replacing Llama 3 70B to cut VRAM and inference cost

VRAM requirements by quantization

Quantization	VRAM required
Q4_K_M (recommended)	14 GB
Q5_K_M	17 GB
Q8_0	26 GB
FP16 (no quantization)	48 GB

VRAM figures include model weights plus a typical 8k KV cache and ~600 MB runtime overhead (Ollama / llama.cpp baseline). Add headroom for higher context lengths.

Published benchmark scores

Benchmark	Score
MMLU	81
GPQA	42.2
HumanEval	84.8

Scores published by the model author or aggregated from public leaderboards. Re-measured monthly by our editorial team.

Strengths

• Quality approaching Llama 3 70B at a third the size
• Low latency relative to peers
• 128k context window
• Strong tool use and agent behavior
• Apache 2.0 license

Limitations

• Needs ~16GB VRAM at Q4, more for higher precision
• Trails Qwen 2.5 Coder on dedicated coding tasks
• No native vision (see Small 3.1 for that)

Architecture & training

Architecture: Dense Transformer · 40 layers · GQA + sliding window

Training: Enriched multilingual corpus, strong focus on FR + scientific English.

Quick start

ollama run mistral-small:24b

Or use the open-source MCP server to query this model from Claude Desktop, Cursor, or any MCP-compatible client.