Which model leads on ARC AI2?

DeepSeek V3 from DeepSeek leads ARC AI2 with a score of 93.7. The median score across 52 tested models is 46.6.

Is ARC AI2 saturated?

No · the top score is 93.7 out of 100 (94%). There is still meaningful room for improvement on ARC AI2.

Does ARC AI2 predict performance on other benchmarks?

Yes · ARC AI2 scores correlate 0.93 with Chatbot Arena Elo · Overall across 6 shared models. Models that do well on ARC AI2 tend to do well on Chatbot Arena Elo · Overall.

How often is ARC AI2 data refreshed?

BenchGecko pulls updates daily. New model scores on ARC AI2 appear as soon as they are published by Epoch AI or the model provider.

Benchmark · KnowledgeSettled

ARC AI2

Name: ARC AI2 Benchmark
Creator: BenchGecko
License: https://creativecommons.org/licenses/by/4.0/

AI2 Reasoning Challenge · tests grade-school level science knowledge with multiple-choice questions requiring reasoning beyond simple retrieval.

Updated 2025-04-15

Models tested

Top score

93.7

DeepSeek V3

Median

46.6

min 0.5

Top-5 spread

σ 2.1

Competitive

The Frontier

Best score over time · one chart, every benchmark

Chart type

Only 5 models have been tested on ARC AI2 · not enough history to compute a frontier yet.

Pink dots = frontier records · 0 totalClick to open model page

Full rankings

52 models tested · sorted by score

#	Model	Score	Price
1	DeepSeek V3· DeepSeek	93.7	$0.26
2	Llama 3.1 405B· Meta	93.7	n/a
3	Qwen2.5 72B Instruct· Alibaba Qwen	92.7	$0.36
4	DeepSeek-V2 (MoE-236B, May 2024)· DeepSeek	89.6	n/a
5	phi-3-medium 14B· Microsoft	88.8	n/a
6	phi-3-small 7.4B· Microsoft	87.6	n/a
7	GPT-3.5 Turbo (older v0613)· OpenAI	83.2	$1.00
8	Mixtral 8x7B· Mistral AI	83.1	n/a
9	Mixtral 8x7B Instruct· Mistral AI	83.1	$0.54
10	Claude Instant· Anthropic	81.7	n/a
11	U Stable Beluga 2· Unknown	81.5	n/a
12	phi-3-mini 3.8B· Microsoft	79.9	n/a
13	Qwen-14B· Alibaba Qwen	79.2	n/a
14	Llama 3 8B Instruct· Meta	77.1	$0.14
15	Llama 3-70B· Meta	77.1	n/a
16	U internlm-20b· Unknown	75.6	n/a
17	Mistral 7B V0.1· Mistral AI	71.5	n/a
18	Phi 2· Microsoft	67.9	n/a
19	Qwen2.5 Coder 32B Instruct· Alibaba Qwen	60.7	$0.66
20	U PaLM 2-L· Unknown	58.9	n/a
21	Falcon-180B· TII	57.1	n/a
22	Qwen2.5 Coder 14B Instruct· Alibaba	54.7	n/a
23	U PaLM 2-M· Unknown	53.2	n/a
24	Qwen2.5 Coder 7B Instruct· Alibaba Qwen	47.9	$0.03
25	Qwen2.5 Coder 7B Instruct· Alibaba	47.9	n/a
26	Llama 2-13B· Meta	47.1	n/a
27	U PaLM 2-S· Unknown	46.1	n/a
28	DeepSeek-Coder-V2-Lite-Base· DeepSeek	43.1	n/a
29	U Nemotron-4 15B· Unknown	40.7	n/a
30	U INTELLECT-1· Unknown	39.4	n/a
31	Qwen-1_8B· Alibaba Qwen	37.6	n/a
32	Qwen2.5-Coder-3B· Alibaba Qwen	37.2	n/a
33	LLaMA-13B· Meta	36.9	n/a
34	U MPT-30B· Unknown	34.1	n/a
35	U Yi 6B· Unknown	33.7	n/a
36	U StarCoder 2 15B· Unknown	29.6	n/a
37	Qwen2.5 Coder 1.5B Instruct· Alibaba	26.9	n/a
38	Phi-1.5· Microsoft	25.9	n/a
39	U vicuna-13b-v1.1· Unknown	24.3	n/a
40	DeepSeek Coder 33B· DeepSeek	22.9	n/a
41	Gemma 2B· Google DeepMind	22.8	n/a
42	U XGen-7B· Unknown	21.6	n/a
43	U Dolly 2.0-12b· Unknown	19.5	n/a
44	U RedPajama-INCITE-7B-Base· Unknown	18.8	n/a
45	open_llama_7b· Meta	18.3	n/a
46	DeepSeek Coder 6.7B· DeepSeek	15.2	n/a
47	CodeQwen1.5-7B· Alibaba Qwen	14.3	n/a
48	Qwen2.5 Coder 0.5B Instruct· Alibaba	12.5	n/a
49	U Baichuan 2-7B· Unknown	10.0	n/a
50	Cerebras-GPT-13B· OpenAI	9.9	n/a
51	U stablelm-tuned-alpha-7b· Unknown	2.7	n/a
52	DeepSeek Coder 1.3B· DeepSeek	0.5	n/a