Methodology — open by default
We distinguish estimates, externally sourced measurements, and our own reproducible experiments. Catalog membership does not mean that we tested a model. The September 12 pilot publishes raw runs for three models on one laptop GPU.
1. Catalog data — how models are indexed
Models are pulled from public sources — Hugging Face, Ollama library, vendor model cards — and reviewed daily by an automated pipeline.
Each model entry exposes: parameters count, quantization variants, license, VRAM requirement at common quants (Q4_K_M, Q5_K_M, Q8_0, FP16), context length, and one-line use case.
The full catalog is exposed under CC BY 4.0 at bestllmfor.com/api/models.json — free to reuse with attribution.
2. Benchmarks — tokens/second on real hardware
Three provenance categories: our own measurement, external measurement with an identified author and protocol, and heuristic estimate.
The September 12, 2026 experiment uses Ollama 0.30.6, context 4096, one warmup and five timed repetitions per model, plus separate synthetic code and French document checks. Raw responses, versions and the full protocol are public. The French checks are not evidence of English-language quality.
Configurator speed uses the model’s low/mid/high catalog value according to GPU tier. CPU fallback uses a size-based multiplier and does not inherit GPU speed. No calibrated error interval is available across all profiles.
3. VRAM requirements — how we estimate
The configurator compares the GPU weight budget with catalog Q4, Q5, Q8 and FP16 thresholds. It selects the highest fitting quantization; it does not calculate the precise KV cache for a chosen context. Leave memory for context and runtime overhead, start at 4096 tokens and verify the actual allocation. On Apple Silicon the GPU budget and system memory share the same physical pool. Memory fit is not a quality recommendation.
Our public calculation code and catalog make these estimates inspectable.
4. Cost calculator — pricing sources
API prices are pulled from vendor pricing pages and cross-checked semestrially. Currently indexed:
- OpenAI — GPT-5, GPT-5 mini
- Anthropic — Claude Opus 4.7, Sonnet 4.6, Haiku 4.5
- Google — Gemini 2.5 Pro, Flash
- DeepSeek — V3.5, R1
- Mistral — Large 2.5, Small 3.5
Self-hosted cost is computed from hardware amortization (you set the months), electricity (you set the €/kWh or $/kWh), GPU TDP, and active hours/day. Output is the monthly break-even token volume.
5. Editorial process — rankings & recommendations
Rankings (e.g. "Best local LLM for coding on RTX 4090") are produced by a hybrid process:
- Filter the catalog by hard constraints (fits in VRAM, license compatible)
- Score remaining candidates against task-specific benchmarks (SWE-bench for coding, MMLU for reasoning, FLORES-200 for translation, etc.)
- Manual review by the author with notes on quirks (regression bugs, prompt sensitivity, refusal rate)
- Public source for every claim: model card, paper, or our own benchmark
No paid placement. No referral kickbacks. If a model leads a ranking, the numbers in the article say why.
6. Reproducing our numbers
Every benchmark page includes:
- Model version + hash (e.g.
qwen3:8b-q4_K_M-2026-04-12) - Runtime version + config (Ollama 0.x.x,
num_ctx,num_thread) - Prompts used (full text in the page footer)
- Date of measurement
If you spot a discrepancy with your own setup, open an issue. We update numbers, not stand-still rankings.