Beta
Leaderboard/Qwen2.5 72B Instruct
Alibaba Qwen

Qwen2.5 72B Instruct

Open Source

by Alibaba Qwen 路 Released 2024-09-19

52.3
avg score
$0.12/1M
Input Price
$0.39/1M
Output Price
33K tokens (~16 books)
Context Window
text
Type

Tested on 15 benchmarks with 52.3% average. Top scores: ARC AI2 (92.7%), MMLU (80.4%), HellaSwag (79.7%).

Benchmark Scores

BenchmarkCategoryScoreBar
ARC AI2knowledge92.7
MMLUknowledge80.4
HellaSwagknowledge79.7
BBHreasoning73.1
TriviaQAknowledge71.9
PIQAknowledge65.2
VideoMMEmultimodal64.7
Winograndeknowledge64.6
MATH level 5math63.2
GeoBenchknowledge62.0
GPQA diamondknowledge32.2
Balrogknowledge16.2
OTIS Mock AIME 2024-2025math8.0
The Agent Companyagentic5.7
OSWorldagentic5.0

Similar Models

U
Yi-34B
unknown
53.0
OpenAIo3
OpenAI
51.3
U
Nemotron-4 15B
unknown
53.4
DeepSeekDeepSeek V3.1
DeepSeek
51.1

Alibaba Qwen Qwen 2.5 Timeline

Qwen2.5 72B Instruct
52.3
$0.12/M in33Kctx15 benchmarks
Qwen2.5 7B Instruct
$0.04/M in(-0.08)33Kctx
Qwen2.5 Coder 32B Instruct
72.6+72.6
$0.66/M in(+0.62)33Kctx5 benchmarks
Qwen2.5 Coder 7B Instruct
61.4-11.2
$0.03/M in(-0.63)33Kctx5 benchmarks
Qwen2.5 VL 32B Instruct
$0.20/M in(+0.17)128Kctx(+95K)
Qwen2.5 VL 72B Instruct
$0.80/M in(+0.60)33Kctx(-95K)