Compile llama.cpp with Metal

#llama.cpp on Mac: what Metal brings

On macOS, the GPU is used through Metal, Apple's graphics and compute API, and llama.cpp uses it directly. The project's README states that Apple Silicon is a first-class citizen, optimized through ARM NEON, Accelerate, and Metal. In practice, this means the default build already produces an engine that uses the GPU, with no option to add, and unified memory avoids any transfer between the CPU and GPU: the model exists in memory only once. The limiting factor on a Mac is therefore this memory's capacity and bandwidth.

#Prerequisites

An Apple Silicon Mac

M1 through M5: that's the target of this guide. Intel-based Macs gain almost nothing from it.

The compilation tools Apple

Install the Command Line Tools with xcode-select --install.

CMake and Git

Available via Homebrew: brew install cmake git.

Memory for the model

The site’s rule of thumb: an 8B in Q4 weighs about 5 GB, a 14B about 9 GB, and a 32B 19 to 20 GB, before context.

xcode-select --install
brew install cmake git

#Install without compiling: Homebrew

If you do not need special compilation options, Homebrew is the fastest route. The llama.cpp installation documentation states that the formula is automatically updated with each new project release. You get the same ready-to-use executables, with Metal.

brew install llama.cpp

Compile it yourself when you want the latest repository version, need to test a branch, or want to modify a compilation option. Otherwise, Homebrew is enough: you save compilation time, and updates are handled with brew upgrade.

#1. Compile with Metal

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp

cmake -B build
cmake --build build --config Release -j $(sysctl -n hw.ncpu)

The build documentation is explicit: on macOS, Metal is enabled by default, and it makes computation run on the GPU. No option needs to be added. The old repository name under Georgi Gerganov's account redirects to the ggml-org organization, where the project now resides. Binaries appear in build/bin, including llama-cli for the terminal and llama-server for the API.

Two useful options: the documentation states that -DGGML_METAL=OFF disables Metal at compile time, and that a binary compiled with Metal can be forced to use the processor with --n-gpu-layers 0, which is useful for comparing speeds.

#2. Run your first model

Recent engines can download the model themselves. The -hf option takes a Hugging Face repository name, with the quantization as a suffix; without a suffix, Q4_K_M is selected by default. For an already downloaded file, use -m with its path.

./build/bin/llama-cli -hf UTILISATEUR/MODELE-GGUF:Q4_K_M

./build/bin/llama-cli -m ~/modeles/mon-modele-Q4_K_M.gguf

The number of layers placed on the GPU is set with -ngl; its default value is auto, which works well on a Mac with unified memory. You can also write -ngl all to load everything. Don't look for a model path in the Ollama directory: its files are stored in an internal format that is not directly usable as a GGUF.

#Serve an OpenAI-compatible API with llama-server

llama-server exposes OpenAI API-compatible routes for chat, responses, and embeddings. By default, it listens on 127.0.0.1, port 8080: it is reachable only from your Mac. To expose it to the network, you must explicitly pass --host, and you should then protect access.

./build/bin/llama-server -m ~/modeles/mon-modele-Q4_K_M.gguf --port 8080

• llama.cpp: what is it, and should you leave Ollama?
• Ollama vs llama.cpp: which one to choose

#4. macOS GPU memory limit

On Apple Silicon, macOS does not give the GPU all of the unified memory. A model that exceeds this share is rejected or spills over to the CPU, even if total memory is sufficient. The engine displays the effective value at startup: look for the line ggml_metal_init: recommendedMaxWorkingSetSize in the llama-cli or llama-server logs.

The sysctl command iogpu.wired_limit_mb lets you raise this limit. It expects a value in megabytes: 61 440 for 60 GB, for example. A llama.cpp repository contributor notes that it must be reapplied at every startup because it is not persistent, and advises against going as high as 100%: the system needs memory for everything not locked by the GPU, and things go badly if you do not leave it enough.

# 56 Go pour le GPU sur un Mac de 64 Go (56 x 1024 = 57344)
sudo sysctl iogpu.wired_limit_mb=57344

# Relancez ensuite le modèle et relisez recommendedMaxWorkingSetSize

#Which model for which unified memory

Unified memory is shared between macOS, your applications, and the model, and the GPU receives only part of it. The site’s guideline places the weights of an 8B model in Q4 at around 5 GB, a 14B model at around 9 GB, and a 32B model at around 19 to 20 GB. The context cache is added on top. The table provides a cautious rule of thumb; the authoritative value is recommendedMaxWorkingSetSize, displayed by the engine.

These orders of magnitude are cautionary estimates, not measurements: a long context, a second model, or a demanding application is enough to change the picture. The dedicated memory guide provides the complete calculation method.

• VRAM and memory: how much do you need for AI?
• Which LLM on a MacBook Pro M4

#5. The options that matter

Flash Attention is enabled automatically by default, so in most cases there is no need to enable it manually. The KV cache can be quantized with -ctk and -ctv, provided Flash Attention is active; compared with f16, q8_0 roughly halves the cache memory usage, at the cost of a slight loss of precision that you should verify for your use cases. The dedicated guide explains the tradeoff in detail.

• Quantize the KV cache to save VRAM
• Flash Attention in llama.cpp

#6. Performance by chip: what public benchmarks measure

QuelLLM does not benchmark these machines. The reference is the “Performance of llama.cpp on Apple Silicon M-series” discussion in the llama.cpp repository, where each contributor runs the same test on a 7B LLaMA in Q4_0. The table below includes a few entries, along with the llama.cpp versions used for each measurement: those for the M1 through M4 chips are from the same version, while the M5 version is newer.

The theoretical ceiling is bandwidth divided by model weight (3.56 GiB, or 3.82 GB). Pro chips reach 71 to 82% of that ceiling, while the Max chip reaches only 58%: beyond a certain point, memory is no longer the only bottleneck, and paying for more bandwidth yields less than the spec sheet suggests.

Three takeaways. Generation follows bandwidth: the M3 Pro, at 150 GB/s, is slower than the M2 Pro at 200 GB/s, despite being a newer chip generation. Max chips, with much higher bandwidth, dominate. Finally, prompt processing took a leap with the M5: 1,620.64 tokens/s for an M5 Pro, versus 439.78 for an M4 Pro with the same number of GPU cores, or 3.7 times faster. This gap matters for long documents and RAG, much less for chat.

A good habit before concluding that a Mac is slow: run the same model with -ngl 0, then with the default value, and compare. The difference between the two shows what the GPU actually contributes on your machine and confirms that computation is going through Metal. Also note the llama.cpp version in use: Metal optimizations evolve quickly, and an old binary can be significantly slower than a recent version.

#Troubleshooting: common errors

• MLX vs. llama.cpp on Mac: who wins in 2026?
• Compile llama.cpp with CUDA
• Which models for 32 GB of memory

• Source: official llama.cpp repository
• Source: llama.cpp build documentation
• Source: public benchmark on Apple Silicon
• Source: discussion about the GPU memory limit of Macs