LM Arena, explained.
TL;DR: LM Arena (lmarena.ai, formerly Chatbot Arena) is a crowdsourced leaderboard where humans vote between two anonymous model answers; millions of votes become an Elo-style rating. It is the best measure of chat preference that exists — and a poor measure of correctness, coding ability, or what runs on your own hardware. Here is how to read it without being misled.
How the arena actually works
You type a prompt, two hidden models answer side by side, you vote for the better reply (or a tie), and only then are the model names revealed. Each vote is a pairwise win/loss. The leaderboard fits a Bradley-Terry model — the same family as chess Elo — over all votes, so a rating gap maps to an expected win rate: about 100 points ≈ 64% expected wins. Separate boards exist per category (overall chat, coding, vision, and others), because a model strong at conversation is not automatically strong at code.
The three biases to keep in mind
Verbosity and style: longer, well-formatted, confident answers win votes even when substance is equal — models tuned for the arena inherit that style premium. Population: voters are self-selected enthusiasts asking chat-style questions, not your production workload. Private variants: research published in 2025 (“The Leaderboard Illusion”) documented providers testing many unreleased variants and shipping the best-scoring one — inflating debut positions. None of this makes the arena useless; it makes it a preference poll, and preference polls reward charm.
How to use it well
Use the arena to shortlist models people actually like talking to, then verify on the axis you care about: task benchmarks for coding (SWE-bench-class), long-context tests for retrieval, and hardware fit for local deployment. Cross-reference at least one static-benchmark leaderboard — style cannot inflate a unit test.
If you run models locally, read this instead
LM Arena ranks models served on provider infrastructure. It cannot tell you what fits in 12GB of VRAM, what license allows commercial use, or what tokens/second you will get. That is what our local LLM leaderboard does — every locally-runnable model, ranked by a transparent composite score with VRAM and license columns, data CC BY 4.0. Pair it with the VRAM calculator to know your budget, and the best Ollama models list to get running today.
Frequently asked questions
What is LM Arena (lmarena.ai)?
LM Arena — formerly Chatbot Arena, started by the LMSYS group at UC Berkeley — is a crowdsourced LLM leaderboard. Visitors chat with two anonymous models side by side, vote for the better answer, and millions of these pairwise votes are converted into an Elo-style rating that ranks models.
Is the LM Arena leaderboard reliable?
It is the best large-scale measure of which model people prefer in open-ended chat — but preference is not correctness. Documented biases include verbosity (longer answers win more votes), style and formatting effects, and providers testing many private variants before releasing a best-scoring one. Treat it as one signal, not a verdict.
Does LM Arena rank local or open-source models?
Open-weight models do appear, but the arena measures chat preference on the providers' own serving stack — not what a model does on your hardware. For self-hosted picks ranked by benchmark scores, VRAM needs and license, see our local LLM leaderboard.
What is the difference between Elo and the arena score?
The arena uses an Elo-style system (formally Bradley-Terry model fitting): every vote is a win/loss between two models, and ratings are fitted so that rating gaps predict win probabilities. A 100-point gap means roughly a 64% expected win rate.
By Mohamed Meguedmi — independent comparator of locally-runnable LLMs, benchmarked on a real RTX 5070 Ti (data CC BY 4.0). BestLLMfor is not affiliated with LM Arena.