Local LLM on Mac M5: MLX now outperforms llama.cpp (figures 2026)

#Why the M5 changes the game for a local LLM on Mac

The M5 chip brings two things that really matter for inference: GPU cores with matrix-multiplication units (Neural Accelerators) integrated into each core, and higher memory bandwidth. LLM inference is primarily a memory-bandwidth problem—each generated token rereads the model's entire weight set. The faster the unified memory, the faster tokens are produced.

This is precisely where Apple's MLX framework has an advantage over llama.cpp. MLX is written to use these new GPU matrix units on the M5 directly, whereas llama.cpp's Metal path remains more generic. The concrete result: on the same machine, with the same model and quantization, MLX generates significantly more tokens per second. The gap, marginal on M3, becomes clear on M5.

#M5 and M5 Max benchmarks: the 2026 figures

Here are approximate figures measured during generation (decode) with MLX, on common models using 4-bit quantization. As always with local inference, these rates vary with context length, the exact model version, and the machine's thermal conditions: treat them as reference points, not guarantees accurate to the token.

• MacBook Pro M5 Max 128 GB: real local AI measurements (2026 benchmark)

M5 (10-core GPU)—8B Q4

~55-70 tokens/s during generation. An 8B model (Qwen 3.5, Granite 4.2, Gemma 4) remains very comfortable on the base chip.

M5 Max — 8B Q4

~230 tokens/s. The Max's higher memory bandwidth makes the difference on small models, where the GPU is never the bottleneck.

M5 Max — 32B Q4

~55–65 tokens/s. A 32B (Qwen 3.8 27B, Devstral) runs at a comfortable reading speed and is highly usable interactively.

M5 Max — 70B Q4

~28 tokens/s. A 70B in Q4 fits in memory with just 48-64 GB of unified RAM and remains fluid for chat and code generation.

The striking point is 70B at ~28 tokens/s on a machine with no dedicated graphics card, one that's quiet and fits in a bag. For comparison, you need a RTX 4090 24 GB (and partial CPU offloading, because a 70B Q4 weighs ~40 GB and doesn't fit in 24 GB of VRAM) to compete on PC — with much more noise and power consumption.

#MLX vs. llama.cpp: where the gap widens

The 30–40% gap in favor of MLX isn’t uniform: it depends on the workload profile. Understanding where it widens helps you know when switching to MLX is really worthwhile.

Generation (decode)

This is where MLX has its clearest advantage on M5, thanks to direct use of the GPU's matrix units. For interactive chat, this is the difference you feel.

Prompt processing (prefill)

MLX retains the advantage, but the gap is somewhat narrower. With very long contexts, both engines become memory-limited.

Model support

llama.cpp remains more universal through the GGUF format; MLX requires a version converted to the MLX format. For popular models, the conversion almost always exists (the mlx-community hub on Hugging Face).

Python integration

MLX is native to Python, making it the obvious choice for prototyping, light fine-tuning, or a custom pipeline. llama.cpp uses bindings.

#Prerequisites and recommended RAM by model

Before installing, calibrate the model size to your unified memory. Here are the realistic tiers on M5 and M5 Max, in Q4.

16 GB (M5)

3B to 8B models are comfortable in Q4. A 14B model works but leaves little room for a large context.

24–32 GB (M5 / M5 Pro)

A 14B handles comfortably; a 32B Q4 is workable at the upper end. The sweet spot for versatile everyday use.

48 GB (M5 Max)

A 32B Q4 is very comfortable, while a 70B Q4 is barely adequate — it's better to target 64 GB for a 70B with context.

64–128 GB (M5 Max)

A smooth 70B Q4 with a large context, or several models loaded in parallel. Power-user territory.

On the software side, you need up-to-date macOS and one of two paths to MLX: LM Studio (which automatically switches to MLX when an MLX version of the model exists) or Ollama since its June 2026 update, detailed just below.

#Ollama’s MLX engine (June 2026)

Historically, Ollama relied on its own engine and llama.cpp, so it used the Metal path on Mac. Since its June 2026 update, Ollama can run models through MLX on Apple Silicon when an MLX version is available—finally bringing the most widely used local AI tool in line with the performance that LM Studio already delivered through MLX.

In practice, this means you get the MLX performance boost without changing your habits: the same Ollama daemon on the default port 11434, the same commands, and the same OpenAI-compatible API. The engine chooses MLX when appropriate and falls back to the standard path otherwise.

#Install and run a model in MLX

Two options, depending on your comfort with the terminal. The most direct way to test MLX without configuration is LM Studio; the most integrable into a stack is Ollama.

1. 01
   1. Update the tool

   Install the latest version of Ollama (June 2026 or newer) or LM Studio. This is required to use the MLX path.
2. 02
   2. Choose a model with an MLX version

   On LM Studio, search prioritizes MLX variants on Apple Silicon. On Ollama, pull a current model: the engine switches to MLX when a converted version exists.
3. 03
   3. Launch and verify throughput

   Ask a long question and watch the displayed tokens/s. Optionally compare it with the same model in GGUF to measure the actual difference on your machine.
4. 04
   4. Adjust quantization if needed

   If the model fills your unified memory, step down a level (Q5 → Q4_K_M) rather than switching tools: memory is the limiting factor, not the engine.

# Vérifier la version d'Ollama (le support MLX date de juin 2026)
ollama --version

# Lancer un modèle : Ollama emprunte MLX quand une version MLX existe
ollama run qwen3:8b

# Le daemon expose l'API compatible OpenAI sur le port par défaut
curl http://localhost:11434/api/tags

On the Python side, MLX is directly suited to scripting, which is useful for benchmarking or integrating inference into a custom pipeline:

# Installer les outils MLX pour LLM (framework Apple, natif Python)
pip install mlx-lm

# Générer avec un modèle converti au format MLX (hub mlx-community)
mlx_lm.generate --model mlx-community/Qwen3-8B-4bit \
  --prompt "Explique la mémoire unifiée du M5 en trois phrases"

#Thunderbolt 5 clusters for 120B+ models

A single Mac, even a well-equipped M5 Max, is limited by its unified memory. To go beyond that — run a 120B+ model or a large MoE — the 2026 approach is to connect multiple Apple Silicon machines over Thunderbolt 5 and split the model across them. Thunderbolt 5 offers significantly more bandwidth than Thunderbolt 4, finally making activation exchange between nodes viable for inference.

The principle is simple: the model is split into layers, each machine hosts part of the weights in memory, and activations move from one node to the next at each token. This lets you combine the unified memory of several Macs to load a model that would fit on none of them individually.

What this unlocks

120B+ models in Q4, or even very large MoE models, by combining, for example, two 128 GB M5 Max systems to approach 256 GB of addressable memory.

The trade-off

Inter-node latency costs tokens/s: a cluster is slower than a single machine that could fit the same model. You use one because no single machine is sufficient, not to gain speed.

The wiring

Thunderbolt 5 is the key: its bandwidth makes activation transfer acceptable. With Thunderbolt 4 or Ethernet, the interconnect becomes the bottleneck again.

#Tips and troubleshooting

“MLX doesn't run faster”

First verify that you are actually using the MLX path (up-to-date tool version, with the model's MLX variant really loaded). A GGUF loaded through the Metal path does not benefit from MLX.

Throughput collapses after a few minutes

This is thermal throttling, especially on MacBook Air (which has no fan). Under sustained load, a MacBook Pro or Mac mini/Studio maintains a more stable throughput.

Model that refuses to load

Unified memory is full. Drop down one model size or one quantization level; close RAM-intensive applications.

Long context that crawls

Prefilling a very long prompt is memory-intensive. Reduce the context window if you don't need it, or accept a slower first token.

#Go further

These guides naturally build on what you just read, from the in-depth comparison to hands-on implementation:

MLX vs. llama.cpp in detail

“MLX vs llama.cpp on M-series Macs: who wins in 2026?” digs deeper into the underlying comparison (model support, quantization, Python integration) beyond the M5 figures alone.

Install Ollama on macOS

“Install Ollama on macOS (Apple Silicon)” covers step-by-step installation and use of unified memory, an essential foundation before targeting MLX.

Choose your quantization

“Choosing your quantization (Q4, Q5, Q8, FP16)” explains the quality/memory tradeoff — the key lever for fitting a model into your unified RAM.