Guide · 2026-08-04

Ollama Update — May 2026 What's New & Worth Trying

Last updated 2026-08-04

May 2026 was Ollama's biggest ship month of the year: a real web search API, llama.cpp merged alongside MLX, and Gemma 4 running ~90% faster on Apple Silicon.

By Mohamed Meguedmi · 9 min read

The month in one paragraph

May 2026 was a heavy ship month across the whole local-AI stack, but Ollama did more than iterate. It closed two long-standing gaps at once: it stopped being a single-backend runtime by merging llama.cpp next to MLX, and it stopped being search-blind by shipping a first-party web search API. Add Codex App support and a fleet of new model weights, and this is the release the team will point back to for the rest of the year. Below we break down what actually changed, what it costs, and which pieces are worth your time today.

The web search API: the real story

The single most important addition is the web search API. Until now, giving a local model live web access meant bolting on a scraper, a SearXNG instance, or a paid search key and wiring it through a tool-calling harness. Ollama now exposes search as a native endpoint, with a generous free tier for individual use and higher rate limits available through Ollama Cloud.

Why this matters more than a changelog line suggests: retrieval is the difference between a model that hallucinates 2024 facts and one that answers today's question. A grounded 8B model routinely beats an ungrounded 70B model on factual, time-sensitive queries. By making search a runtime primitive, Ollama lets a small local model punch far above its parameter count — the classic small-model-plus-tools pattern that we track in our benchmarks.

Practical read: if you build agents, the web search API alone justifies the upgrade. It removes the flakiest part of the local-agent stack — the search plumbing — and replaces it with something the runtime maintains.

0.30.x: llama.cpp joins MLX under one roof

The other structural change is the backend story. Earlier 2026 builds leaned heavily on MLX for Apple Silicon performance, which was fast but narrowed GGUF and cross-platform coverage. The 0.30.0 release integrates llama.cpp alongside MLX, so the runtime can pick the right engine per model and per platform.

In practice that means:

• Broader model coverage. GGUF quants that lagged behind on MLX-only builds now load cleanly.
• No workflow change. ollama pull and ollama run behave exactly as before; backend selection is automatic.
• Fewer "unsupported architecture" errors on newly released weights, because llama.cpp typically merges new architectures fast.

If you upgrade from a pre-0.30 build, expect a one-time re-index on first launch and slightly larger disk use as both backends' shims are present. Nothing in the model store needs to be re-pulled.

MTP speedups: Gemma 4 gets ~90% faster

Multi-token prediction (MTP) is the performance headline. Instead of decoding one token per forward pass, MTP predicts several and verifies them, cutting wall-clock latency on the same hardware. Per the official Ollama releases, Gemma 4 now generates tokens nearly 90% faster on average on Apple Silicon across a coding-agent benchmark by leveraging MTP.

This is a free lunch: same weights, same quality, roughly half the latency on supported models. It is most visible on the interactive, bursty workloads where latency is felt — coding agents, chat, and tool loops — and less dramatic on long single-shot generations.

Change	What it affects	Who benefits most
Web search API	Live retrieval / grounding	Agent and RAG builders
llama.cpp + MLX dual backend	Model coverage + speed	Everyone on cross-platform hardware
MTP decoding	Token latency	Apple Silicon coding-agent users
Codex App support	Editor / IDE integration	Developers in agentic editors

Five new models worth pulling

May's model drops span the full hardware range. Below is a quick reference. VRAM figures are approximate for the noted quant and include KV-cache headroom for a modest context; see our cost calculator to translate these into hardware and cloud spend.

Model	Best-fit quant	Approx. VRAM	Sweet spot
SmolLM2 1.7B	Q8_0	~2–3 GB	Laptops, edge, drafting/speculative decode
Granite 3.3 8B	Q4_K_M	~6–7 GB	Enterprise RAG, long-context summarization
GLM-5.1 (mid)	Q4_K_M	~20–24 GB	General reasoning + coding on a single 24GB card
Kimi K2.5	Q4_K_M	~48 GB+	Long-context agents, document workflows
Nemotron Ultra	Q4_K_M	Multi-GPU / server	Frontier open reasoning, batch serving

Our short verdict on the five: GLM-5.1 is the best all-round pull for a single 24GB card and the one most readers should try first. Granite 3.3 8B is the pragmatic enterprise pick — permissively licensed, strong at retrieval-grounded tasks, and cheap to run (its model card lives on Hugging Face). SmolLM2 is the sleeper: at 1.7B it is an excellent draft model for speculative decoding and runs anywhere. Kimi K2.5 and Nemotron Ultra are server-class — impressive, but only worth it if you have the VRAM budget. Cross-reference the full lineup in our model catalog.

Codex App support and the IDE angle

Ollama also added Codex App support this month, extending its reach into agentic editors. The pattern is the same one we keep recommending: run a fast local model for the tight, private, high-frequency edits, and reserve a cloud model for the occasional heavy reasoning step. With MTP cutting local latency and the web search API adding grounding, the local half of that split is now genuinely competitive for day-to-day coding.

Ollama Cloud: is Pro worth $20?

The web search free tier is aimed at individuals; sustained or high-volume use pushes you toward Ollama Cloud. Independent comparisons this month put Ollama Cloud Pro at the top of the ~$20/month class for access to large open models, with the caveat that the free tier hits a wall quickly on the biggest models.

Tier	Price	Best for	Watch-out
Local only	$0 (your hardware)	Privacy, unlimited tokens, offline	You provide the VRAM
Cloud Free	$0	Trying big models, light search	Rate limits bite fast on large models
Cloud Pro	~$20/mo	Frequent big-model + search use	Recurring cost; data leaves your machine

Our position: if your hardware runs GLM-5.1 or Granite 3.3 comfortably, stay local and use the free search tier — you get privacy and zero marginal cost. Pro earns its price only if you regularly need frontier-class open models you cannot host, or search volume beyond the free allowance. Model the break-even with our cost calculator before subscribing.

Should you upgrade? The verdict

Yes. This release adds two genuinely new capabilities (search, dual backend) and one large free performance gain (MTP), and it does so without breaking the standard pull-and-run workflow. The only friction is a one-time re-index on first launch. For agent and RAG builders it is close to mandatory; for everyone else it is a clean, low-risk win.

You are…	Upgrade?	First thing to try
Building agents / RAG	Yes — immediately	Native web search API
On Apple Silicon	Yes	Gemma 4 with MTP
Single 24GB GPU user	Yes	GLM-5.1 Q4_K_M
Enterprise / RAG on a budget	Yes	Granite 3.3 8B Q4_K_M
Edge / laptop	Yes	SmolLM2 1.7B as a draft model

All the benchmark data behind our verdicts is available through the free BestLLMfor public API (CC BY 4.0), and you can query it directly from your own agents via our open-source MCP server. For methodology on how we measure latency and VRAM, see our methodology page, and compare runtimes on the best-of rankings.