Which model leads on Winogrande?

Llama 3.1 405B from Meta leads Winogrande with a score of 78.4. The median score across 53 tested models is 46.8.

Is Winogrande saturated?

No · the top score is 78.4 out of 100 (78%). There is still meaningful room for improvement on Winogrande.

Does Winogrande predict performance on other benchmarks?

Yes · Winogrande scores correlate 0.94 with PIQA across 20 shared models. Models that do well on Winogrande tend to do well on PIQA.

How often is Winogrande data refreshed?

BenchGecko pulls updates daily. New model scores on Winogrande appear as soon as they are published by Epoch AI or the model provider.

Benchmark · ReasoningSettled

Winogrande

Name: Winogrande Benchmark
Creator: BenchGecko
License: https://creativecommons.org/licenses/by/4.0/

WinoGrande · large-scale commonsense reasoning benchmark where models must resolve ambiguous pronouns in carefully constructed sentence pairs.

Updated 2025-04-15

Models tested

Top score

78.4

Llama 3.1 405B

Median

46.8

min 3.0

Top-5 spread

σ 1.5

Settled

The Frontier

Best score over time · one chart, every benchmark

Chart type

Only 5 models have been tested on Winogrande · not enough history to compute a frontier yet.

Pink dots = frontier records · 0 totalClick to open model page

Full rankings

53 models tested · sorted by score

#	Model	Score	Price
1	Llama 3.1 405B· Meta	78.4	n/a
2	Claude 3 Opus· Anthropic	77.0	n/a
3	GPT-4 (older v0314)· OpenAI	75.0	$30.00
4	GPT-4 Turbo· OpenAI	75.0	$10.00
5	Falcon-180B· TII	74.2	n/a
6	DeepSeek-V2 (MoE-236B, May 2024)· DeepSeek	72.6	n/a
7	DeepSeek V3· DeepSeek	70.4	$0.26
8	Llama 3 70B Instruct· Meta	67.0	$0.51
9	Llama 3-70B· Meta	67.0	n/a
10	U PaLM 2-L· Unknown	66.0	n/a
11	Qwen2.5 72B Instruct· Alibaba Qwen	64.6	$0.36
12	GPT-3.5 Turbo (older v0613)· OpenAI	63.2	$1.00
13	phi-3-medium 14B· Microsoft	63.0	n/a
14	phi-3-small 7.4B· Microsoft	63.0	n/a
15	Qwen2.5 Coder 32B Instruct· Alibaba Qwen	61.6	$0.66
16	U PaLM 2-M· Unknown	58.4	n/a
17	Falcon 2 11B· TII	56.6	n/a
18	U Nemotron-4 15B· Unknown	56.0	n/a
19	U PaLM 2-S· Unknown	55.8	n/a
20	Mixtral 8x7B· Mistral AI	54.4	n/a
21	Mixtral 8x7B Instruct· Mistral AI	54.4	$0.54
22	Qwen2.5 Coder 14B Instruct· Alibaba	53.6	n/a
23	Llama 3 8B Instruct· Meta	51.4	$0.14
24	Mistral 7B V0.1· Mistral AI	50.6	n/a
25	Claude 3 Sonnet· Anthropic	50.2	n/a
26	Claude 3 Haiku· Anthropic	48.4	$0.25
27	Phi-1.5· Microsoft	46.8	n/a
28	LLaMA-13B· Meta	46.0	n/a
29	DeepSeek-Coder-V2-Lite-Base· DeepSeek	45.8	n/a
30	Qwen2.5 Coder 7B Instruct· Alibaba Qwen	45.8	$0.03
31	Qwen2.5 Coder 7B Instruct· Alibaba	45.8	n/a
32	Llama 2-13B· Meta	45.6	n/a
33	U Yi 6B· Unknown	42.6	n/a
34	U MPT-30B· Unknown	42.0	n/a
35	phi-3-mini 3.8B· Microsoft	41.6	n/a
36	U vicuna-13b-v1.1· Unknown	41.6	n/a
37	Qwen2.5-Coder-3B· Alibaba Qwen	34.8	n/a
38	open_llama_7b· Meta	34.0	n/a
39	U INTELLECT-1· Unknown	31.6	n/a
40	Gemma 2B· Google DeepMind	30.8	n/a
41	U XGen-7B· Unknown	29.8	n/a
42	U StarCoder 2 15B· Unknown	28.6	n/a
43	U RedPajama-INCITE-7B-Base· Unknown	27.6	n/a
44	DeepSeek Coder 33B· DeepSeek	24.0	n/a
45	U Dolly 2.0-12b· Unknown	23.6	n/a
46	Cerebras-GPT-13B· OpenAI	21.6	n/a
47	Qwen2.5 Coder 1.5B Instruct· Alibaba	21.4	n/a
48	CodeQwen1.5-7B· Alibaba Qwen	19.6	n/a
49	DeepSeek Coder 6.7B· DeepSeek	15.2	n/a
50	Qwen2.5 Coder 0.5B Instruct· Alibaba	9.6	n/a
51	Phi 2· Microsoft	9.4	n/a
52	DeepSeek Coder 1.3B· DeepSeek	6.6	n/a
53	U stablelm-tuned-alpha-7b· Unknown	3.0	n/a