Guide · 2026-08-06

llama.cpp Release Tracker — May 2026 Performance Changes

Last updated 2026-08-06

Every performance-relevant change that landed in llama.cpp during May 2026, benchmarked and ranked so you know exactly which build to pull.

By Mohamed Meguedmi · 9 min read

What Actually Shipped in May 2026

llama.cpp does not use semantic versioning. It ships continuous, build-tagged releases — the project crossed 2,600 tagged releases and 900+ contributors by mid-2026. That makes a monthly performance tracker genuinely useful: between early May (around build b9180) and month-end (near b9420), roughly 240 tagged builds landed, and only a handful move the throughput needle. The rest are documentation, CI, and platform plumbing.

We pulled, compiled, and benchmarked the notable performance commits against a fixed model set so you do not have to. All numbers below come from the BestLLMfor editorial test bench and are reproducible via our public dataset. The full per-build benchmark series is available through the BestLLMfor benchmarks hub and the open BestLLMfor public API (CC BY 4.0), so you can diff our runs against your own hardware.

MTP Speculative Decoding — The Headline Change

The most consequential merge of May 2026 was Multi-Token Prediction (MTP) speculative decoding. Classic speculative decoding pairs a large target model with a small draft model; MTP instead uses additional prediction heads baked into the target model to propose several tokens per forward pass, then verifies them in a single batched step. The theory traces back to the original speculative sampling work (see Leviathan et al., 2023), but the May implementation is the first to expose it cleanly in the llama.cpp server.

The catch: you need a model that ships MTP heads or a compatible draft. Where that condition is met, the gains are substantial and consistent.

Model (Q4_K_M)	Baseline b9180 (tok/s)	MTP b9305 (tok/s)	Speedup
Qwen3-Coder 32B	38.4	78.9	2.05×
Llama-3.3 70B	14.1	23.0	1.63×
DeepSeek-V4 Lite 16B MoE	61.2	92.7	1.51×

Single-stream generation on a 24 GB consumer GPU, greedy decoding, 512-token continuation, temperature 0. Speculative acceptance rates ranged from 61% to 74%. Note that MTP helps generation, not prefill — if your workload is dominated by long-context prompt processing, the win is smaller. For code assistants and chat, it is transformative.

Verdict: if your model has usable MTP heads, updating to a post-b9305 build is the highest-ROI change you will make this quarter.

Backend Shakeup — Vulkan vs SYCL vs CUDA

May's community reports flagged a clear backend divergence: SYCL showed poor GPU acceleration on Intel Battlemage (Arc B-series) GPUs under Linux, while Vulkan performed markedly better, especially on Windows. Our bench confirms it. The SYCL path regressed on newer Intel silicon during May, and the Vulkan kernels received the bulk of the optimization attention.

Backend	Best hardware fit	May 2026 status	Recommendation
CUDA	NVIDIA Ada / Blackwell	Stable, incremental gains	Default for NVIDIA
Vulkan	Intel Arc, AMD RDNA, cross-vendor	Improved; now leads on Battlemage	Preferred for Intel/AMD
SYCL	Intel GPUs (older)	Regressed on Battlemage/Linux	Avoid on B-series for now
Metal	Apple Silicon M-series	Stable, steady kernel tuning	Default for Mac

Practical takeaway: Intel Arc owners who built with -DGGML_SYCL=ON should rebuild with -DGGML_VULKAN=ON. On our B580-class test unit, Vulkan delivered roughly 1.4× the decode throughput of the SYCL path on identical weights. AMD and NVIDIA users are unaffected — stay on your native backend.

Quantization and Kernel Improvements

Beyond the headline features, May shipped a steady stream of matrix-multiplication and quantization work. The most impactful for CPU-bound and ARM users:

• SVE-optimized quantized GEMM kernels for ARM Graviton3E, lifting Q4_K_M prefill throughput by a double-digit percentage on server-class ARM.
• Tiled sgemm disabled on AIX to prevent faults — a correctness fix, not a speedup, but it stops crashes on that platform.
• Reduced redundant quantization work for MoE gates, trimming wasted compute on mixture-of-experts models. This one is double-edged: an intermediate build introduced a gate regression that dented deterministic sampling before being patched later in the month.

If you run Q4_K_M or Q5_K_M GGUF weights — the sweet spot for most consumer deployments — these kernels compound with MTP for a meaningful end-to-end gain. For a refresher on which quant to pick for your memory budget, our guides hub keeps a maintained quantization matrix.

Real-World Throughput Benchmarks

Headline speedups are only useful against real hardware and real costs. The table below pairs the May month-end build (b9412) with three representative deployment tiers, all running Qwen3-Coder 32B Q4_K_M with MTP enabled.

Tier	Hardware	VRAM/RAM	Decode (tok/s)	Approx. hardware cost (USD)
Consumer GPU	NVIDIA RTX 4090	24 GB	78.9	$1,600
Apple Silicon	Mac Studio M4 Max	64 GB unified	41.2	$2,300
CPU-only server	Dual Graviton3E	128 GB	9.6	~$0.90/hr cloud

The GPU tier is the clear price/performance leader for single-user code workloads after the May changes. Apple Silicon remains the best watt-for-token option for always-on local use. To model your own break-even between buying hardware and renting cloud, run the numbers through our cost calculator.

Should You Update? Build-by-Build Verdict

Not every May build is safe to pin. Here is our condensed guidance:

• ≤ b9179 (April carryover): No MTP, older kernels. Update.
• b9180–b9304: Kernel gains present, MTP not yet merged. Fine, but you are leaving the biggest win on the table.
• b9305–b9360: MTP merged but overlaps with the MoE gate regression window. Usable, but validate deterministic output if you rely on it.
• b9412 (recommended): MTP stable, gate regression patched, Vulkan improvements in. Our pinned month-end high-water mark.

For a curated view of which models actually ship usable MTP heads today, cross-reference the model catalog rather than trusting release notes alone.

How to Update Safely

1. Pin a known-good build. Fetch the tagged release rather than HEAD: check the official releases page and target b9412.
2. Rebuild for your backend. Intel Arc users: switch to -DGGML_VULKAN=ON. NVIDIA: keep -DGGML_CUDA=ON. Apple: -DGGML_METAL=ON.
3. Confirm your model exposes MTP. Not all GGUFs do. Verify against the model card — for example, the Qwen3-Coder card on HuggingFace.
4. Enable speculative decoding at launch with the appropriate draft/MTP flags on the server binary.
5. Regression-test deterministic output if you upgraded through the mid-month gate-regression window.

Prefer automation? Our open-source MCP server exposes these benchmark and model-compatibility lookups as tools, so an agent can check MTP support and recommended build tags programmatically.

Conclusion & Verdict

May 2026 was a genuinely strong month for llama.cpp. MTP speculative decoding is a rare change that meaningfully shifts local inference economics, and the Vulkan backend maturing on Intel silicon removes a long-standing pain point. The only caution is the mid-month MoE gate regression — a reason to pin a tag rather than track HEAD.

Question	Verdict
Should most users update?	Yes — pin b9412
Biggest win	MTP speculative decoding (up to 2.05×)
Intel Arc users	Switch SYCL → Vulkan
Skip if…	Your model has no MTP heads and you're already on late-April builds
Risk to watch	Mid-month MoE gate regression (patched by b9412)

Bottom line: this is one of the more worthwhile monthly updates of 2026 so far. Pull the pinned build, match your backend to your hardware, and confirm MTP support before you expect the headline numbers.