Beta
Leaderboard/Llama 3.1 70B Instruct
Meta

Llama 3.1 70B Instruct

Open Source

by Meta 路 Released 2024-07-23

26.1
avg score
$0.40/1M
Input Price
$0.40/1M
Output Price
131K tokens (~66 books)
Context Window
text
Type

Tested on 7 benchmarks with 26.1% average. Top scores: MMLU (73.5%), MATH level 5 (36.7%), Balrog (27.9%).

Benchmark Scores

BenchmarkCategoryScoreBar
MMLUknowledge73.5
MATH level 5math36.7
Balrogknowledge27.9
GPQA diamondknowledge25.6
WeirdMLcoding9.0
The Agent Companyagentic6.9
OTIS Mock AIME 2024-2025math3.5

Similar Models

Mistral AIMistral Large
Mistral AI
25.8
DeepSeekDeepSeek Coder 33B
DeepSeek
25.4
MicrosoftPhi-1.5
Microsoft
27.2
MetaLlama 4 Maverick
Meta
28.0

Meta Llama 3.1 Timeline

Llama 3.1 70B Instruct
26.1
$0.40/M in131Kctx7 benchmarks
Llama 3.1 8B Instruct
28.7+2.6
$0.02/M in(-0.38)16Kctx(-115K)8 benchmarks
Llama 3.1-405B
49.3+20.6
N/AN/Actx15 benchmarks