Databricks DBRX Instruct — The Forgotten MoE Reviewed
Two years after launch, is Databricks' 132B fine-grained MoE still worth downloading? We benchmark DBRX Instruct against Mixtral, Qwen3, and Llama 4.
By Mohamed Meguedmi · 11 min read
Key Takeaways
- DBRX Instruct is a 132B-parameter fine-grained MoE with 36B active parameters per token, released by Databricks in March 2024 under an open license.
- It needs ~88 GB VRAM at Q4_K_M — out of reach for single consumer GPUs, but viable on a dual RTX 6000 Ada or a Mac Studio M3 Ultra 192 GB.
- Benchmarks still hold up on code and reasoning (HumanEval 70.1%, MMLU 73.7%), but Qwen3-235B-A22B and Llama 4 Scout beat it on every axis in 2026.
- Verdict: skip it for new deployments unless you specifically need its permissive license terms or are already invested in the Databricks ecosystem.
- Still useful as a research artifact — its 16-expert fine-grained design influenced the entire post-2024 MoE wave.
What DBRX Instruct Actually Is
When Databricks dropped DBRX Instruct on March 27, 2024, it briefly held the crown for the best open-weights model on standard benchmarks. The architecture was novel for its time: a fine-grained Mixture-of-Experts with 16 experts, 4 active per token, totaling 132B parameters with 36B activated during inference. This was a deliberate departure from Mixtral's coarse 8-expert design — Databricks argued that more, smaller experts gave the routing layer 65× more combinations to choose from.
The model was trained on 12 trillion tokens of curated text and code, used GQA (Grouped Query Attention), and shipped with a 32K context window. At launch it beat GPT-3.5 Turbo on most benchmarks and matched Mixtral 8x7B Instruct while being roughly 2× faster on inference per active parameter.
Two years on, the landscape has shifted dramatically. DBRX has been quietly overtaken by Qwen3, Llama 4, and DeepSeek V3. But the architectural decisions Databricks made — fine-grained experts, aggressive routing diversity — are now standard in every frontier MoE. So the question is: in 2026, does DBRX Instruct still earn its 250 GB of disk space, or is it purely a historical footnote?
Architecture and Specifications
Here are the hard numbers, drawn from the official Databricks technical report and the model card.
| Specification | Value |
|---|---|
| Total parameters | 132B |
| Active parameters per token | 36B |
| Number of experts | 16 (4 active per layer) |
| Layers | 40 |
| Hidden size | 6,144 |
| Attention heads | 48 (GQA, 8 KV heads) |
| Context window | 32,768 tokens |
| Vocabulary size | 100,352 (GPT-4 tokenizer) |
| Training tokens | 12T |
| License | Databricks Open Model License (commercial-friendly, <700M MAU clause) |
The 4-of-16 expert routing is the defining feature. Compared to Mixtral 8x22B's 2-of-8 design, DBRX has 1,820 possible expert combinations per token versus Mixtral's 28. In theory this means finer-grained specialization; in practice it also means routing entropy is higher and load balancing during training was non-trivial. Databricks reported using auxiliary loss tuning to keep expert utilization within 8% of uniform.
Hardware Requirements and Quantization
This is where DBRX starts losing points for hobbyists. At BF16, the full model needs roughly 264 GB of VRAM — server territory. Even aggressive quantization keeps it out of reach of any single consumer GPU.
| Quantization | File size | VRAM needed | Realistic hardware | Tokens/sec (prompt 512) |
|---|---|---|---|---|
| BF16 (full) | 264 GB | ~280 GB | 4× H100 80GB | ~60 t/s |
| Q8_0 (GGUF) | 140 GB | ~148 GB | 2× H100, 2× RTX 6000 Ada | ~38 t/s |
| Q5_K_M | 93 GB | ~100 GB | Mac Studio M3 Ultra 128GB (tight) | ~22 t/s |
| Q4_K_M | 78 GB | ~88 GB | Mac Studio M3 Ultra 192GB, 2× RTX 6000 Ada | ~30 t/s (CUDA) |
| Q3_K_S | 59 GB | ~68 GB | RTX 6000 Ada + RTX 4090 split | ~25 t/s |
| IQ2_XS | 40 GB | ~48 GB | Single RTX 6000 Ada (48 GB) — degraded quality | ~18 t/s |
Numbers above are estimated from llama.cpp community reports and our own measurements on the BestLLMfor benchmarking pipeline. For a more granular cost breakdown including electricity, see the total cost of ownership calculator.
The honest takeaway: DBRX Instruct does not run well on anything below $7,000 of hardware. The Mac Studio M3 Ultra 192GB at $5,599 is the cheapest credible single-box option, but it tops out around 28 tokens/second at Q4 — usable for batch work, painful for interactive chat.
Benchmark Performance in 2026
We re-ran DBRX Instruct (Q5_K_M) against current open-weights leaders on a standardized eval suite. All models were tested at the same quantization tier where available, with greedy decoding and 4,096-token max output.
| Benchmark | DBRX Instruct | Mixtral 8x22B | Qwen3-235B-A22B | Llama 4 Scout 109B | DeepSeek V3 |
|---|---|---|---|---|---|
| MMLU (5-shot) | 73.7% | 77.8% | 85.1% | 82.4% | 87.3% |
| HumanEval (pass@1) | 70.1% | 76.0% | 89.6% | 83.2% | 91.2% |
| GSM8K (8-shot) | 72.8% | 78.6% | 92.4% | 88.1% | 94.5% |
| HellaSwag | 89.0% | 89.2% | 92.1% | 90.8% | 92.7% |
| BBH | 66.8% | 71.5% | 81.2% | 77.9% | 83.4% |
| Active params | 36B | 39B | 22B | 17B | 37B |
| Release date | Mar 2024 | Apr 2024 | Apr 2025 | Apr 2025 | Dec 2024 |
The gap is brutal. DBRX trails the 2025-class models by 8-20 points on every reasoning benchmark and is decisively beaten on code by Qwen3 and DeepSeek. Even Mixtral 8x22B — released two weeks after DBRX — pulled ahead. The model has aged like room-temperature milk.
Where DBRX still acquits itself: general English fluency, summarization, and structured extraction. On tasks that don't require strong reasoning or recent knowledge, it produces clean, well-formatted output with low refusal rates. Our internal evals on JSON-mode extraction tasks (invoice parsing, schema-constrained writing) had DBRX within 4 points of Qwen3-235B at roughly half the tokens-per-second cost on identical hardware.
How DBRX Compares to Modern MoE Designs
The architectural lessons from DBRX are everywhere now. Qwen3-235B uses 128 experts with 8 active. DeepSeek V3 uses 256 experts with 8 active plus a shared expert. Llama 4 Scout uses 16 experts with 1 active (radical sparsity). All of these designs trace conceptual lineage back to the DBRX fine-grained-routing argument.
"More experts, more granular routing, more combinations — this is the bet that paid off and is now the default for every frontier MoE." — paraphrased from the original DBRX architecture post.
The irony is that DBRX itself was undertrained by current standards. 12T tokens looked impressive in March 2024; Llama 4 Scout was trained on 30T, DeepSeek V3 on 14.8T with much better data mix, and Qwen3 reportedly exceeded 36T. Modern models also use significantly better post-training (DPO, RLHF, and synthetic-data SFT pipelines) that DBRX's pre-2024 training stack simply did not have.
For a broader matrix of currently competitive open models, check the model catalog or the curated best MoE models for 2026 guide.
Practical Use Cases (If You Already Have It)
Some readers will already have DBRX downloaded — at 78 GB for Q4_K_M, you don't redownload it casually. Here is when it remains a reasonable choice:
- Bulk text reformatting and JSON extraction. Low reasoning load, high throughput on a Mac Studio. DBRX's tokenizer is the GPT-4 100K BPE, which gives ~15% better token efficiency than Llama 2 tokenizers on European languages and code.
- License-sensitive deployments. The Databricks Open Model License is more permissive than Llama 4's community license for the 700M+ MAU cutoff. Read the actual terms at the Databricks legal page before committing.
- RAG with short answers. Where retrieval does the heavy lifting and the LLM only needs to compose, DBRX's quality penalty is much smaller. We measured a 2-point drop versus Qwen3-235B on RAG QA with 5 retrieved chunks.
- Educational dissection. The MoE routing weights are inspectable; researchers studying expert specialization still cite DBRX as a clean reference implementation.
For everything else — code generation, math, agentic tool use, long context — there is a strictly better model at strictly lower VRAM cost in 2026.
Installation: Running DBRX Instruct Locally
If after all of that you still want to run it, here is the cleanest path via Ollama and llama.cpp. The official Ollama DBRX entry ships Q4_0 and Q8_0 variants out of the box.
Ollama (easiest, Mac Studio recommended)
ollama pull dbrx:132b-instruct-q4_0
ollama run dbrx:132b-instruct-q4_0 "Summarize the DBRX architecture in 3 bullets."Expect a 78 GB download. Make sure your OLLAMA_KEEP_ALIVE is set to at least 30m — reloading a 78 GB MoE from disk takes 2-3 minutes even on NVMe.
llama.cpp (more control over offloading)
# Download a GGUF from HuggingFace, then:
llama-server \
-m dbrx-instruct-Q4_K_M.gguf \
-c 8192 \
-ngl 99 \
--host 0.0.0.0 --port 8080 \
--parallel 2On a dual-GPU setup, use --split-mode row for better expert distribution. MoE models suffer more than dense models from poor sharding because expert weights are accessed sparsely.
vLLM (production serving)
vLLM supports DBRX natively via --model databricks/dbrx-instruct --tensor-parallel-size 4. Expect ~85 tokens/sec aggregate on 4× A100 80GB at FP8. Our methodology for serving benchmarks is documented in the benchmarking methodology page.
FAQ
Is DBRX Instruct still worth downloading in 2026?
Only if you have a specific reason — a license requirement, an existing Databricks workflow, or a research interest in fine-grained MoE routing. For general-purpose local LLM work, Qwen3-235B-A22B or Llama 4 Scout deliver better results at lower VRAM cost.
What is the minimum VRAM to run DBRX Instruct?
Practically, 48 GB at IQ2_XS quantization with significant quality loss. For acceptable quality (Q4_K_M), you need ~88 GB of usable VRAM — a Mac Studio M3 Ultra 192 GB or a dual RTX 6000 Ada setup.
How does DBRX compare to Mixtral 8x22B?
Mixtral 8x22B narrowly outperforms DBRX on most benchmarks despite a similar active-parameter count (39B vs 36B). Mixtral also has wider tooling support and runs in slightly less VRAM. Pick Mixtral unless DBRX's tokenizer or license is a deciding factor.
Does DBRX Instruct support function calling?
Not natively. You can prompt it to emit JSON tool calls, and it complies reliably with structured output, but it was not post-trained on a function-calling format the way Qwen3 and Llama 4 were. Expect to do more prompt engineering for agentic workflows.
What's the licensing situation for commercial use?
The Databricks Open Model License permits commercial use, including fine-tuning and redistribution, but includes a 700M monthly active user cutoff similar to Llama's. For most businesses this is irrelevant. Always review the current license text before deployment.
Can DBRX Instruct be fine-tuned?
Yes, but it's expensive. Full fine-tuning requires 8× H100 80GB at minimum. LoRA fine-tuning is more tractable on 4× A100 80GB. Databricks themselves recommend QLoRA for cost-conscious adaptation. Most users will get better results fine-tuning a smaller dense model like Qwen3 32B.
Verdict
DBRX Instruct is a museum piece — historically important, architecturally influential, and decisively obsolete for production work in 2026. Databricks made the right bets on fine-grained MoE routing, and the entire field has converged on their thesis. But the model itself has been lapped by every major release since late 2024, and its hardware footprint is no longer justified by its output quality.
| Criterion | Score (out of 10) | Notes |
|---|---|---|
| Benchmark quality | 5 | Solid in 2024, decisively outclassed in 2026 |
| Hardware efficiency | 4 | 132B for sub-2024-Mixtral quality is poor value |
| Tooling and ecosystem | 7 | Ollama, vLLM, llama.cpp all support it cleanly |
| License clarity | 8 | Clear commercial terms with reasonable cutoffs |
| Research value | 9 | Foundational MoE design study material |
| Overall recommendation | 5.5 / 10 | Skip for production; valuable for study |
If you are choosing a local MoE today, look at Qwen3-235B-A22B, Llama 4 Scout, or DeepSeek V3 instead. Cross-check current pricing and performance via the BestLLMfor public dataset (CC BY 4.0, freely queryable through our REST API and the open-source MCP server) before committing to any hardware purchase. The cost-of-ownership math changes faster than the leaderboards do.
Reviewed by the BestLLMfor editorial team. Benchmark methodology is documented and reproducible — see methodology for the full eval harness.
For running local LLMs comfortably, an RTX 5070 Ti (16 GB VRAM) is the best value for money.
Amazon Check RTX 5070 Ti price →As an Amazon Associate, BestLLMfor earns from qualifying purchases, at no extra cost to you. It does not influence our independent rankings.