Mixture of Experts (MoE) Explained: Why a 120B Model Can Outrun a 27B

What a mixture of experts is

A standard ("dense") transformer pushes every token through every parameter. Most of those parameters live in the feed-forward layers. An MoE model splits each feed-forward layer into a set of parallel sub-networks, the experts, typically 8 to 256 of them, and adds a small router that scores the experts for each token and sends the token to the best few, commonly 2 to 8. The outputs are combined, and the token moves to the next layer, where a different set of experts may be chosen.

The attention layers and embeddings are usually shared by all tokens; only the feed-forward capacity is sparse. The idea goes back decades, and was adapted to large language models in work such as Google's Switch Transformer (2021). It reached open-weight users with Mistral AI's Mixtral 8x7B at the end of 2023. As of September 20, 2026, 78 of the 239 models in the BestLLMfor catalog are MoE, including most of the largest.

What the "experts" are not

The name suggests a committee of specialists: one expert for code, one for French, one for biology. That is not what happens. Routing is learned, per token and per layer, and the specializations that emerge are mostly low-level and hard to interpret: punctuation, certain token types, syntactic roles. You cannot remove "the math expert," and nothing in an MoE resembles separate models taking turns. It is one network with conditional wiring.

Total vs active parameters

Model	Total parameters	Active per token	Memory at 4-bit	License
gpt-oss 20B	21B	3.6B	13 GB	Apache 2.0
Qwen 3 30B-A3B	30B	3B	19 GB	Apache 2.0
Qwen3-Coder 30B-A3B	30B	3.3B	19 GB	Apache 2.0
GLM 4.7 Flash	31B	≈ 3B	19 GB	MIT
Qwen 3.6 35B-A3B	35B	3B	21 GB	Apache 2.0
Mixtral 8x7B	47B	≈ 13B	26 GB	Apache 2.0
Llama 4 Scout	109B	17B	65 GB	Llama 4 Community
gpt-oss 120B	117B	5.1B	70 GB	Apache 2.0
Qwen 3.5 122B-A10B	122B	10B	73 GB	Apache 2.0
Qwen 3 235B-A22B	235B	22B	142 GB	Apache 2.0
DeepSeek R1	671B	37B	400 GB	MIT

From the BestLLMfor model catalog, September 20, 2026.

Mixtral's name is a classic trap: "8x7B" is not 56B. The experts share attention layers, so the total is 47B, and with two experts active per token roughly 13B parameters do the work.

Memory follows the total, speed follows the active

Generating a token means reading from memory every weight that token uses. A dense model reads all of them. An MoE reads the shared layers plus only the selected experts. Since generation speed on local hardware is set by memory bandwidth divided by the bytes read per token, fewer active parameters translate almost directly into more tokens per second.

Model	Type	Memory at 4-bit	Catalog speed rating, mid-range GPU	High-end GPU
Qwen 3 8B	Dense, 8B	5 GB	35 tok/s	90 tok/s
Qwen 3 32B	Dense, 32B	19 GB	12 tok/s	30 tok/s
Qwen 3 30B-A3B	MoE, 3B active	19 GB	40 tok/s	100 tok/s
Llama 3.3 70B	Dense, 70B	40 GB	6 tok/s	20 tok/s
gpt-oss 120B	MoE, 5.1B active	70 GB	35 tok/s	90 tok/s

Speed ratings are the estimated tiers used in the BestLLMfor catalog, assuming the model fits in fast memory. They are planning figures, not measurements of your machine.

Read the two bold rows against their neighbors. The 30B MoE costs the same memory as the dense 32B and runs like an 8B. The 120B MoE needs nearly twice the memory of a dense 70B and is rated several times faster. We observed the same pattern in measured data: on a MacBook Pro M5 Max with 128 GB, gpt-oss 120B generated at 75–79 tokens per second while a dense 27B model managed 37 (full benchmark).

Is a 30B MoE as smart as a 30B dense model?

No, and not as weak as a 3B either. A long-standing rule of thumb places an MoE's capability near the geometric mean of its total and active parameters: for 30B total and 3B active, around 9–10B dense-equivalent. It is a heuristic, not a law, and recent MoE models often beat it. The honest summary: at equal memory, a dense model is usually somewhat stronger; at equal speed, the MoE is far stronger. Which one is "better" depends on whether your constraint is VRAM or patience.

Why labs build them

• Training cost scales with active parameters. A lab can train a model with the knowledge capacity of hundreds of billions of parameters for the compute bill of a few tens of billions.
• Serving cost per token is lower, for the same reason, once the model is spread across enough GPUs.
• Capacity is cheap to add. More experts means more stored knowledge without slowing generation.

The price is memory, plus engineering complexity: routers must be trained to spread load evenly, and serving needs fast interconnect between GPUs.

Running an MoE on consumer hardware

The MoE trade suits two kinds of machines unusually well.

• Large unified memory. A Mac with 64–128 GB, or a Ryzen AI Max+ machine, has plenty of capacity and moderate bandwidth. Dense 70B models crawl there; MoE models with 3–10B active parameters fly. See AI hardware and best LLMs for a 128 GB Mac.
• A modest GPU plus plenty of system RAM. Keep attention layers and the KV cache on the GPU and leave expert weights in RAM. Because only a few experts are touched per token, the penalty is far smaller than offloading a dense model.

# llama.cpp: everything on the GPU except the experts of the first 24 layers
llama-server -m gpt-oss-120b-Q4_K_M.gguf -ngl 99 --n-cpu-moe 24 -c 16384

Raise --n-cpu-moe until the model fits in VRAM, then lower it as far as memory allows. With 64 GB of system RAM and a 16–24 GB GPU, this makes 100B-class MoE models usable at reading speed, which no dense model of that size can manage. Fast system RAM matters here: dual-channel DDR5 helps measurably. For the memory arithmetic, see what is VRAM and the VRAM calculator. Catalog data on this page is open through the BestLLMfor public API (CC BY 4.0) and our MCP server.

When to choose an MoE

Your constraint	Pick
8–12 GB of VRAM, little system RAM	A dense 8B–14B model; MoE totals will not fit
16 GB of VRAM	gpt-oss 20B (13 GB): MoE speed in a small footprint
24 GB of VRAM	A 30B-A3B-class MoE for speed and agents; a dense 27B–32B when answer quality matters more than pace
64–128 GB of unified memory	MoE, clearly: gpt-oss 120B or a 100B-class Qwen
GPU plus 64 GB+ of DDR5	A large MoE with expert offload