Guide · 2026-07-17

How to Run Llama on Apple Silicon with MLX — Native Performance

Last updated 2026-07-17

MLX has won the Apple Silicon performance race. Here is the exact, tested path to running Llama at native speed on any M-series Mac.

By Mohamed Meguedmi · 9 min read

Why MLX beats llama.cpp for Llama on Apple Silicon

MLX is Apple's open-source array framework, released in December 2023 and purpose-built for machine learning on Apple Silicon. The distinction that matters for inference is memory model: MLX was designed around unified memory from day one, so tensors are shared between CPU and GPU with zero copies. llama.cpp reaches the GPU through a Metal backend layered onto a CPU-first design, which adds overhead that compounds at high token rates.

The practical result is speed. Independent testing puts MLX at 20–87% faster generation than llama.cpp for models under 14B parameters on the same hardware, with the largest gains on small models where per-token overhead dominates. llama.cpp still wins in two cases: cross-platform portability (it runs everywhere) and very long contexts, where its KV-cache handling remains competitive. If your target is a Mac and the model is Llama, MLX is the verdict.

There is a second reason to prefer MLX: it is Python-native. If your stack already speaks Python, MLX integrates directly instead of shelling out to a compiled binary, which simplifies fine-tuning, batching, and custom sampling.

Hardware and unified-memory requirements

The single number that decides what you can run is unified memory. On Apple Silicon, RAM is shared between the CPU, GPU, and Neural Engine, so the model weights plus KV cache must fit inside your total memory alongside macOS itself. Budget roughly 3–4 GB for the system before counting model weights.

Llama model	Quantization	Weights on disk	Peak unified memory	Minimum Mac
Llama 3.2 3B Instruct	4-bit	~1.8 GB	~3 GB	M1, 8 GB
Llama 3.1 8B Instruct	4-bit	~4.5 GB	~6 GB	M1/M2, 16 GB
Llama 3.1 8B Instruct	8-bit	~8.5 GB	~11 GB	M-series, 16 GB
Llama 4 Scout 17B (MoE)	4-bit	~60 GB	~68 GB	M3/M4 Max/Ultra, 96 GB+
Llama 3.3 70B Instruct	4-bit	~40 GB	~46 GB	M2/M3 Max, 64 GB

Two rules follow from this table. First, 16 GB machines are comfortable up to 8B at 4-bit and workable at 8-bit, but nothing larger. Second, running Llama 3.3 70B in practice means a 64 GB Mac, and Llama 4 Scout's mixture-of-experts weights only make sense on 96 GB or higher. For a full cross-model breakdown, see our benchmarks hub, and use the cost calculator to compare buying a higher-memory Mac against cloud inference over your expected token volume.

Install MLX and run Llama in under five minutes

The entire toolchain is one Python package, mlx-lm, maintained by the MLX team. It bundles the runtime, a CLI, a chat interface, and an OpenAI-compatible server.

1. Confirm your Mac and macOS. You need Apple Silicon (M1 or newer) and macOS 13.5+. Run uname -m and confirm it prints arm64.
2. Set up Python 3.9+. A clean virtual environment avoids dependency conflicts:

   python3 -m venv mlx-env
   source mlx-env/bin/activate

3. Install mlx-lm.

   pip install --upgrade mlx-lm

4. Generate your first tokens. This pulls a pre-quantized Llama from the mlx-community on Hugging Face and runs it:

   mlx_lm.generate \
     --model mlx-community/Llama-3.1-8B-Instruct-4bit \
     --prompt "Explain unified memory in one paragraph." \
     --max-tokens 256

5. Start an interactive chat when you want a REPL instead of one-shot generation:

   mlx_lm.chat --model mlx-community/Llama-3.1-8B-Instruct-4bit

That is the whole install. There is no build step, no Metal flag, and no separate GGUF download tool, because MLX loads its own safetensors-based format directly from the mlx-community model hub. If you need a model that is not yet converted, mlx_lm.convert --hf-path meta-llama/Llama-3.1-8B-Instruct -q quantizes it locally.

MLX vs llama.cpp: real throughput numbers

The table below reflects generation speed for Llama at 4-bit across common Mac configurations. Numbers are single-stream tokens/sec on short prompts; expect them to drop as context grows. Treat them as directional, since macOS version, thermal state, and background load all move the result.

Mac chip	Model	MLX (tok/s)	llama.cpp Metal (tok/s)	MLX advantage
M2 (16 GB)	Llama 3.1 8B 4-bit	~42	~28	+50%
M3 Max (36 GB)	Llama 3.1 8B 4-bit	~98	~72	+36%
M4 Max (48 GB)	Llama 3.1 8B 4-bit	~115	~84	+37%
M2 Ultra (128 GB)	Llama 3.3 70B 4-bit	~20	~17	+18%
M3 Max (64 GB)	Llama 3.3 70B 4-bit	~9	~8	+13%

The pattern is consistent with the theory: the smaller the model, the larger MLX's edge, because per-token dispatch overhead is a bigger share of the work. At 70B the two runtimes converge, since the workload is dominated by raw memory bandwidth, which both backends saturate. If a model comfortably fits your memory and sits below 14B, MLX is the clear pick. Our full methodology for these measurements is documented at /methodology/.

Choosing the right quantization

Quantization is the lever that most affects both quality and memory, and it matters more than which chip generation you have. MLX supports 3-bit through 8-bit and mixed schemes. Our recommendations for Llama:

• 4-bit (Q4): the default for chat, summarization, and general assistants. Quality loss versus 8-bit is small and the memory savings are large.
• 6-bit: the best balance for coding and structured output, where 4-bit occasionally produces malformed JSON or off-by-one logic.
• 8-bit: reserve for agentic pipelines and tool calling, where accumulated errors across many steps degrade reliability. Use it when you have the memory headroom.
• 3-bit: only when a model would otherwise not fit. Expect a visible quality drop; a smaller model at 4-bit is usually a better trade.

A concrete example: on a 32 GB Mac, Llama 3.1 8B at 8-bit (~11 GB) leaves ample room and gives near-lossless quality, so there is no reason to run 4-bit there. On a 16 GB Mac the same 8-bit build is tight once your editor and browser are open, so 4-bit is the safer default. Browse converted variants for any model in the model catalog.

Serving Llama as an OpenAI-compatible API

To wire Llama into existing apps, MLX ships a server that speaks the OpenAI Chat Completions format. Start it with one command:

mlx_lm.server --model mlx-community/Llama-3.1-8B-Instruct-4bit --port 8080

Point any OpenAI client at http://localhost:8080/v1 with a placeholder API key, and requests route to your local Llama. This is the fastest way to migrate a prototype from a paid API to on-device inference: no code changes beyond the base URL. For continuous batching and vision-language models, community servers built on MLX extend the same interface.

All of the throughput and memory figures on BestLLMfor are also available programmatically through our public API, licensed CC BY 4.0, and through our open-source MCP server, so you can pull the latest Llama-on-MLX benchmarks directly into your own tooling rather than copying tables by hand.

Verdict

For running Llama on a Mac, MLX is the recommended runtime in every case where the model fits in unified memory and portability is not a requirement. It is faster, simpler to install, and better integrated with Python than the llama.cpp Metal path. Reach for llama.cpp only when you need the same binary to run on Linux or Windows, or when you are pushing extremely long contexts.

Use case	Recommended runtime	Suggested Llama build
Mac-only chat assistant	MLX	Llama 3.1 8B 4-bit
Coding / structured output on Mac	MLX	Llama 3.1 8B 6-bit
Agentic / tool-calling pipeline	MLX	Llama 3.3 70B 4-bit (64 GB+)
Cross-platform deployment	llama.cpp	GGUF Q4_K_M
Very long context (100k+)	llama.cpp	GGUF Q4_K_M