What does MMLU-PRO measure?

MMLU-PRO is a knowledge benchmark in the BenchGecko catalog. 78 AI models have been tested on it. Scores range from 0.3 to 52.6 out of 100.

Which model leads on MMLU-PRO?

Qwen2-72B from Alibaba Qwen leads MMLU-PRO with a score of 52.6. The median score across 78 tested models is 24.2.

Is MMLU-PRO saturated?

No · the top score is 52.6 out of 100 (53%). There is still meaningful room for improvement on MMLU-PRO.

Does MMLU-PRO predict performance on other benchmarks?

Yes · MMLU-PRO scores correlate 0.93 with CMMLU across 5 shared models. Models that do well on MMLU-PRO tend to do well on CMMLU.

How often is MMLU-PRO data refreshed?

BenchGecko pulls updates daily. New model scores on MMLU-PRO appear as soon as they are published by Epoch AI or the model provider.

Benchmark · KnowledgeSettled

MMLU-PRO

Name: MMLU-PRO Benchmark
Creator: BenchGecko
License: https://creativecommons.org/licenses/by/4.0/

Updated 2025-10-17

Models tested

Top score

52.6

Qwen2-72B

Median

24.2

min 0.3

Top-5 spread

σ 1.5

Settled

The Frontier

Best score over time · one chart, every benchmark

Chart type

Only 21 models have been tested on MMLU-PRO · not enough history to compute a frontier yet.

Pink dots = frontier records · 0 totalClick to open model page

Full rankings

78 models tested · sorted by score

#	Model	Score	Price
1	Qwen2-72B· Alibaba Qwen	52.6	n/a
2	Qwen2.5 32B Instruct· Alibaba	51.9	n/a
3	Qwen2.5 72B Instruct· Alibaba Qwen	51.4	$0.36
4	HA Qwen2.5 72B Instruct Abliterated· HuiHui AI	50.4	n/a
5	Phi 4· Microsoft	48.3	$0.07
6	Llama 3.3 70B Instruct· Meta	48.1	$0.10
7	Llama 3.1 70B Instruct· Meta	47.9	$0.40
8	Qwen2.5 14B Instruct· Alibaba	43.2	n/a
9	DeepSeek R1 Distill Llama 8B· DeepSeek	41.6	n/a
10	R1 Distill Llama 70B· DeepSeek	41.6	$0.80
11	Hermes 3 70B Instruct· nousresearch	41.4	$0.70
12	Meta Llama 3 8B· Meta	41.2	n/a
13	DeepSeek R1 Distill Qwen 32B· DeepSeek	41.0	n/a
14	R1 Distill Qwen 32B· DeepSeek	41.0	$0.29
15	DeepSeek R1 Distill Qwen 14B· DeepSeek	40.7	n/a
16	WizardLM-2 8x22B· Microsoft	40.0	$0.62
17	D Dolphin 2.9.1 Yi 1.5 34b· DPHN	39.1	n/a
18	Gemma 2 27B· Google DeepMind	38.4	$0.65
19	Qwen2.5 Coder 32B Instruct· Alibaba Qwen	37.9	$0.66
20	U Yi 6B· Unknown	37.9	n/a
21	Qwen2.5 7B Instruct· Alibaba Qwen	36.5	$0.10
22	GLM 4 32B · z-ai	34.9	$0.10
23	Qwen2 VL 7B Instruct· Alibaba	34.4	n/a
24	Phi 3 Mini 4k Instruct· Microsoft	33.6	n/a
25	Phi 3.5 Mini Instruct· Microsoft	32.9	n/a
26	Qwen2.5 Coder 14B Instruct· Alibaba	32.7	n/a
27	Phi 4 Mini Instruct· Microsoft	32.6	$0.08
28	Phi 4 Mini Instruct· Microsoft	32.6	n/a
29	Gemma 2 9B· Google DeepMind	31.9	$0.03
30	Qwen2 7B Instruct· Alibaba	31.6	n/a
31	Magnum v4 72B· anthracite-org	31.4	$3.00
32	Llama 3.1 8B Instruct· Meta	30.9	$0.02
33	Meta Llama 3 8B Instruct· Meta	29.6	n/a
34	Qwen2.5 Coder 7B Instruct· Alibaba Qwen	26.1	$0.03
35	Qwen2.5 Coder 7B Instruct· Alibaba	26.1	n/a
36	U Stable Beluga 2· Unknown	25.9	n/a
37	Llama 3.1 405B· Meta	25.7	n/a
38	Qwen2.5 3B Instruct· Alibaba	25.1	n/a
39	Llama 3.2 3B Instruct· Meta	24.4	$0.05
40	Llama 3-70B· Meta	24.0	n/a
41	LLaMA-13B· Meta	23.1	n/a
42	Hermes 2 Pro - Llama-3 8B· nousresearch	22.8	$0.14
43	Mistral 7B V0.1· Mistral AI	22.4	n/a
44	Gemma 2B· Google DeepMind	21.6	n/a
45	Qwen2.5 1.5B Instruct· Alibaba	20.0	n/a
46	Mistral 7B Instruct V0.2· Mistral AI	19.1	n/a
47	Phi 2· Microsoft	18.1	n/a
48	Llama 3 8B Instruct· Meta	17.8	$0.14
49	Gemma 2 2b It· Google DeepMind	17.2	n/a
50	Qwen2 1.5B Instruct· Alibaba	16.7	n/a
51	Llama 3.2 3B Instruct (free)· Meta	16.5	$0.00
52	Mistral 7B Instruct v0.1· Mistral AI	15.7	$0.11
53	Falcon-180B· TII	15.4	n/a
54	U StarCoder 2 15B· Unknown	15.0	n/a
55	DeepSeek R1 Distill Qwen 7B· DeepSeek	14.7	n/a
56	Gemma 2 2b· Google DeepMind	13.5	n/a
57	L Vicuna 7b V1.5· LMSYS	12.7	n/a
58	Llama 2 7b Hf· Meta	9.6	n/a
59	Llama 3.2 1B Instruct· Meta	8.2	$0.03
60	Qwen2 0.5B· Alibaba	8.0	n/a
61	Qwen2.5 0.5B Instruct· Alibaba	8.0	n/a
62	Phi-1.5· Microsoft	7.7	n/a
63	Llama 2 7b Chat Hf· Meta	7.6	n/a
64	Qwen2 0.5B Instruct· Alibaba	5.9	n/a
65	U MPT-30B· Unknown	2.3	n/a
66	U RedPajama-INCITE-7B-Base· Unknown	2.2	n/a
67	QwQ 32B· Alibaba Qwen	2.2	$0.15
68	DeepSeek R1 Distill Qwen 1.5B· DeepSeek	2.1	n/a
69	D Distilgpt2· DistilBERT	2.1	n/a
70	Gpt2 Medium· OpenAI	2.0	n/a
71	Gpt2· OpenAI	1.7	n/a
72	Gpt2 Large· OpenAI	1.6	n/a
73	U INTELLECT-1· Unknown	1.3	n/a
74	Pythia 160m· eleutherai	1.3	n/a
75	HF SmolLM2 135M Instruct· Hugging Face TB	1.3	n/a
76	T TinyLlama 1.1B Chat V1.0· TinyLlama	1.1	n/a
77	HF SmolLM2 135M· Hugging Face TB	1.1	n/a
78	Gpt Neo 125m· eleutherai	0.3	n/a