AI Benchmarks Explained (2026): What Each One Measures and Who Leads

An AI benchmark is a fixed set of tasks used to score a model’s ability, so that
different models can be compared on the same test. There is no single benchmark that
settles which model is best — each measures something narrow, and several of the well-known ones
are now close to saturated. The practical approach is to read the two or three that match your
actual workload, and treat any single headline number with suspicion.

Who currently leads

Ranked by the Artificial Analysis Intelligence Index, a composite of several independent
evaluations rather than one test. The last column is the one most comparisons leave out: score
divided by blended price, which is what you are really buying.

The benchmarks that matter in 2026

MMLU (Massive Multitask Language Understanding)

Multiple-choice questions across 57 subjects, from elementary mathematics to professional
law. For years it was the default headline number, and that is now the problem: frontier models
cluster so tightly at the top that differences between them fall inside the noise. MMLU is still
useful for placing a small open model against a large one, and close to useless for separating
two frontier models. Treat a 2026 MMLU claim as a floor check, not a ranking.

GPQA (Graduate-Level Google-Proof Q&A)

Written by domain PhDs in biology, physics and chemistry, and deliberately designed so that
searching the web does not help. It answers a different question from MMLU: not “has the model
read a lot” but “can it reason through a genuinely hard problem in a technical field”. Because
it resists both memorisation and retrieval, it has aged better than MMLU as a discriminator.

SWE-bench Verified

Real bug reports from real GitHub repositories, scored on whether the model’s patch makes the
project’s own tests pass. The Verified subset is the human-validated portion, filtered to remove
tasks that were broken or impossible — which is why you should check which SWE-bench a
vendor is quoting. This is the single most decision-relevant benchmark if you are choosing a
model for a coding agent, because the task and the grading are both authentic. Claude Sonnet 5
reports 85.2% on SWE-bench Verified and 63.2% on the harder SWE-bench Pro.

Terminal-Bench

Agentic tasks carried out in a real terminal: install something, diagnose a failure, script a
fix. It measures a different skill from writing a patch — tolerance for multi-step work where
the model must read its own error output and recover. Claude Sonnet 5 reports 80.4% on
Terminal-Bench 2.1. If your use case is an autonomous agent rather than an inline code
assistant, this and OSWorld tell you more than SWE-bench does.

OSWorld

Computer use in a real desktop environment — open applications, click through interfaces,
complete a task a person would do with a mouse. Scores here are far lower than on text
benchmarks across the board, which is an honest signal about how immature computer-use agents
still are. Claude Sonnet 5 reports 81.2% on OSWorld-Verified; NVIDIA’s Nemotron 3 Nano Omni,
a 30B open model, reports 47.4% on OSWorld, which is a reasonable result for its size.

ARC-AGI

Abstract visual reasoning puzzles built so that pattern-matching from training data does not
transfer. Each task is a handful of grid transformations the model must infer from a couple of
examples. ARC-AGI is the closest thing the field has to a test of genuine generalisation rather
than recall, which is why progress on it is slow and why jumps are treated as significant.
Claude Opus 5 is reported at roughly three times the next-best model on ARC-AGI 3.

Humanity’s Last Exam

Expert-authored questions across many disciplines, written specifically to resist the
saturation that overtook MMLU. The design goal is a benchmark that stays hard as models improve,
so scores are low by construction and small gains are meaningful. Read it as a frontier
discriminator, not as a measure of usefulness for ordinary work.

Artificial Analysis Intelligence Index

Not a test but a composite, combining several independent evaluations into one score. That is
its value: any single benchmark can be targeted, and a blend is much harder to game. It is the
scale used in the table above. Claude Opus 5 is the current leader at 61, with Claude Fable 5 at
60 and GPT-5.6 Sol at 59 — and Kimi K3 at 57, the strongest open-weight result recorded, sitting
within four points of the best closed model.

How to read a benchmark claim

Ask who ran it. A vendor-reported score on its own launch page and an
independent evaluation are different kinds of evidence. Independent leaderboards apply the same
harness to every model; a vendor picks its own conditions.

Check for contamination. If a benchmark’s questions predate a model’s
training cut-off and exist publicly, some of them are likely in the training data. This is the
main reason older benchmarks inflate over time and why newer ones are built to be
“Google-proof” or held out entirely.

Watch for a moved goalpost. “SWE-bench” without “Verified”, an unnamed subset,
a different number of attempts, or a score with tool use compared against one without — these
are the usual ways a comparison stops being like for like.

Beware saturation. When every serious model scores above 90 on a test, that
test has stopped ranking anything. Differences of a point or two at the top of a saturated
benchmark are noise, not signal.

Price is part of the score. A model two points better and six times more
expensive is not better for most workloads. This is why the table above carries a score-per-dollar
column, and why the price-performance
index exists.

Which benchmark should you actually care about?

Building a coding agent: SWE-bench Verified first, Terminal-Bench second.
Ignore MMLU entirely.

Building a computer-use or desktop agent: OSWorld, then Terminal-Bench.
Expect the absolute numbers to be low.

Technical or scientific question answering: GPQA, and Humanity’s Last Exam if
your questions are genuinely expert-level.

General assistant or chat: a composite index is more informative than any
single test, because no one benchmark resembles the spread of things real users ask.

Choosing a small model to self-host: benchmarks matter less than fit. Check
what will physically run on your hardware with the
VRAM calculator first, then compare
benchmark scores only among the models that fit.

Published benchmark results in our database

Frequently asked questions