Noometry

Model comparison

GPT-4.5 vs Llama 3.1-8B

GPT-4.5 is the stronger model overall, scoring 36.9 to 22.8 on the Noometry Index.

Last verified . 22 shared benchmarks.

GPT-4.5 OpenAI

36.9

Rank #200 Confirmed

Llama 3.1-8B Meta

22.8

Rank #347 Confirmed

Summary

  • They share 22 benchmarks with published results for both. GPT-4.5 scores higher in 8 categories and Llama 3.1-8B in 1 category; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where GPT-4.5 leads 57.1 to 29.9.
  • The biggest single-benchmark swing is MATH Level 5: 78.6% for GPT-4.5 and 22.9% for Llama 3.1-8B.
  • Llama 3.1-8B has downloadable open weights; the other is API-only.

Side by side

GPT-4.5Llama 3.1-8B
ProviderOpenAIMeta
Noometry Index36.922.8
Released2025-02-272024-07-23
WeightsProprietaryOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.05
Output $ / M tokens—$0.08
Results tracked4243

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4.5 leads

GPT-4.5: 41.8 (#102), Llama 3.1-8B: 19.9 (#335)

BenchmarkGPT-4.5Llama 3.1-8B
WeirdML39.4%1.7%
LMArena Coding13971195
Aider Polyglot44.9%—
SciCode—13.2%
BigCodeBench Instruct—32.8%
LiveBench Coding75.2%—
BigCodeBench Complete—40.5%
HumanEval+—62.8%
MBPP+—55.6%

Agentic & Tool Use GPT-4.5 leads

GPT-4.5: 27.6 (#92), Llama 3.1-8B: 22.3 (#123)

BenchmarkGPT-4.5Llama 3.1-8B
Berkeley Function Calling Leaderboard—25.8%
Cybench17.5%—
BALROG—15.1%

Reasoning Too close to call

GPT-4.5: 14.1 (#320), Llama 3.1-8B: 14.5 (#316)

BenchmarkGPT-4.5Llama 3.1-8B
LMArena Hard Prompts14041175
Epoch Capabilities Index136.74116.48
ARC-AGI-20.8%—
SimpleBench34.5%—
ARC-AGI-110.3%—
CritPt—0%
Chess Puzzles—0%
EnigmaEval3.2%—
LiveBench Reasoning71.1%—
DTBench—50.9%
LiveBench Data Analysis64.3%—
LMCA—5.4%
ForecastBench61.7—
LiveBench69%—
PIQA—81.2%

Math GPT-4.5 leads

GPT-4.5: 31.0 (#200), Llama 3.1-8B: 9.5 (#311)

BenchmarkGPT-4.5Llama 3.1-8B
OTIS Mock AIME 2024-202537.8%1.7%
LMArena Math14121179
MATH Level 578.6%22.9%
Omni-MATH—13.7%
LiveBench Math69.3%—
GSM8K—82.4%

Knowledge GPT-4.5 leads

GPT-4.5: 32.4 (#205), Llama 3.1-8B: 8.0 (#302)

BenchmarkGPT-4.5Llama 3.1-8B
GPQA Diamond68.7%27%
LMArena Expert13941144
Humanity's Last Exam5.4%—
MMLU-Pro—40.6%
Confabulations13.6%—
GPQA (HELM)—24.7%
BoolQ—82.8%
MMLU—56.1%

Multimodal Not comparable

GPT-4.5: 37.0 (#68), Llama 3.1-8B: —

BenchmarkGPT-4.5Llama 3.1-8B
LMArena Vision1195—
VPCT45%—

Multilingual GPT-4.5 leads

GPT-4.5: 52.7 (#78), Llama 3.1-8B: 34.2 (#244)

BenchmarkGPT-4.5Llama 3.1-8B
LMArena Non-English14131148
LMArena Chinese14211151
LMArena French14171176
LMArena German14571144
LMArena Japanese14161061
LMArena Korean13921053
LMArena Russian14191158
LMArena Spanish—1169

Instruction Following GPT-4.5 leads

GPT-4.5: 72.8 (#129), Llama 3.1-8B: 59.1 (#253)

BenchmarkGPT-4.5Llama 3.1-8B
LMArena Instruction Following14041159
LiveBench Instruction Following72.3%—
IFEval—74.3%

Long Context GPT-4.5 leads

GPT-4.5: 40.4 (#150), Llama 3.1-8B: 35.8 (#233)

BenchmarkGPT-4.5Llama 3.1-8B
LMArena Longer Query14061182
Fiction.LiveBench63.9%—

Writing & Preference GPT-4.5 leads

GPT-4.5: 57.1 (#129), Llama 3.1-8B: 29.9 (#285)

BenchmarkGPT-4.5Llama 3.1-8B
LMArena Text14171187
LMArena Creative Writing13941154
EQ-Bench Creative Writing1258713
LMArena Multi-Turn14441172
Short-Story Creative Writing75.6%—
WildBench—68.7%
LiveBench Language61.5%—

Frequently asked questions

Is GPT-4.5 better than Llama 3.1-8B?

GPT-4.5 is the stronger model overall, scoring 36.9 to 22.8 on the Noometry Index.

Is GPT-4.5 or Llama 3.1-8B better for coding?

GPT-4.5 scores higher on coding benchmarks: 41.8 versus 19.9 in the Noometry coding category.

How many benchmarks do GPT-4.5 and Llama 3.1-8B share?

22 benchmarks have published results for both models. GPT-4.5 has 42 scored results on Noometry and Llama 3.1-8B has 43.

Related comparisons

Go deeper