Best local LLM for agents
Last updated 2026-08-28 · Page updated 2026-08-31
Top 7 open-source picks for autonomous agents and tool use, ranked by benchmark performance and real-world fit. Updated monthly.
Verdict (August 2026): Qwen 3 30B-A3B is our current top pick for autonomous agents and tool use (30B params · Apache 2.0). The full ranking and per-model reasoning follow.
This ranking is built from real specs — parameter count, VRAM at 4-bit quantization, license terms, and available benchmark scores — pulled from BestLLMfor's tracked catalog of 239 models, not vendor marketing. Data reflects the catalog as of 2026-08-28; see our full methodology for how models are scored and re-ranked.
Qwen 3 30B-A3B
Alibaba's Qwen 3 MoE with 30B total and just 3B active parameters, supporting hybrid thinking mode. MMLU 81.4, AIME24 80.4, 100+ languages, Apache 2.0.
gpt-oss 20B
OpenAI's compact open-weight MoE with 3.6B active out of 21B total parameters. Matches o3-mini on a laptop-class GPU under Apache 2.0.
ERNIE 4.5 21B-A3B Thinking
Baidu's compact reasoning MoE with 3B active parameters out of 21B total. Fast inference thanks to the small active set, with Chinese-language strength.
Kanana 2 30B-A3B Thinking
Kakao's agentic 30B MoE (3B active) with native hybrid thinking and Korean-first training. Apache 2.0 with MLA attention and 131k context.
DeepSeek R1 Distill 32B
The 32B DeepSeek R1 distill — the best accessible open-weight reasoner we've tested. Explicit chain-of-thought, MIT-licensed, runs on a single 24GB GPU.
Qwen 3 32B
Alibaba's 32B dense flagship with thinking mode, scoring 65.5 on MMLU-Pro and 39.8 on SuperGPQA. The strongest general-purpose Qwen 3 dense model before stepping up to the MoE.
QwQ 32B
Alibaba's dedicated 32B reasoner, trained with reinforcement learning rather than distillation. Hits 79.5 on AIME24 and 90.6 on MATH-500 — a direct Apache-licensed alternative to DeepSeek R1.
Which hardware should you buy to run Qwen 3 30B-A3B?
To run Qwen 3 30B-A3B locally at Q4, you need ~19 GB of VRAM. The best value for this today is a GMKtec EVO-X2 64GB (Ryzen AI Max+ 395 mini PC) (64 GB unified memory, half the price of an RTX 5090).
As an Amazon Associate, BestLLMfor earns from qualifying purchases, at no extra cost to you. It does not influence our independent rankings.
Frequently asked questions
What is the best local LLM for autonomous agents and tool use?
Qwen 3 30B-A3B tops this ranking — a 30B model, licensed under Apache 2.0, needing about 19 GB of VRAM at Q4 quantization. See the full list below for the runner-ups and how they compare.
How much VRAM do I need to run Qwen 3 30B-A3B?
At Q4 quantization, Qwen 3 30B-A3B needs about 19 GB of VRAM and fits comfortably on a single 24 GB GPU.
Which of these models fit an 16 GB GPU?
At Q4 quantization, gpt-oss 20B, ERNIE 4.5 21B-A3B Thinking fit within 16 GB of VRAM.
Are the models on this autonomous agents and tool use list free for commercial use?
Licenses across this list include Apache 2.0, MIT. Check the specific license of each model on its catalog page before deploying commercially, as terms vary by author.
What context window do these models support?
Context windows on this list range from 32k to 128k tokens, depending on the model.