Noometry

Model comparison

GPT-4.5 vs Llama 3.1-405B

GPT-4.5 is the stronger model overall, scoring 36.9 to 30.4 on the Noometry Index.

Last verified . 26 shared benchmarks.

GPT-4.5 OpenAI

36.9

Rank #200 Confirmed

Llama 3.1-405B Meta

30.4

Rank #279 Confirmed

Summary

  • They share 26 benchmarks with published results for both. GPT-4.5 scores higher in 8 categories and Llama 3.1-405B in 1 category; 9 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where GPT-4.5 leads 57.1 to 39.1.
  • The biggest single-benchmark swing is MATH Level 5: 78.6% for GPT-4.5 and 49.8% for Llama 3.1-405B.
  • Llama 3.1-405B has downloadable open weights; the other is API-only.

Side by side

GPT-4.5Llama 3.1-405B
ProviderOpenAIMeta
Noometry Index36.930.4
Released2025-02-272024-07-23
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked4242

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4.5 leads

GPT-4.5: 41.8 (#102), Llama 3.1-405B: 32.7 (#257)

BenchmarkGPT-4.5Llama 3.1-405B
WeirdML39.4%21.4%
LMArena Coding13971291
Aider Polyglot44.9%—
LiveBench Coding75.2%—

Agentic & Tool Use GPT-4.5 leads

GPT-4.5: 27.6 (#92), Llama 3.1-405B: 20.9 (#131)

BenchmarkGPT-4.5Llama 3.1-405B
Cybench17.5%7.5%
TheAgentCompany—7.4%

Reasoning Llama 3.1-405B leads

GPT-4.5: 14.1 (#320), Llama 3.1-405B: 16.4 (#293)

BenchmarkGPT-4.5Llama 3.1-405B
SimpleBench34.5%23%
LMArena Hard Prompts14041270
Epoch Capabilities Index136.74128.76
ForecastBench61.759.9
ARC-AGI-20.8%—
Kagi LLM Benchmark—45%
ARC-AGI-110.3%—
EnigmaEval3.2%—
LiveBench Reasoning71.1%—
DTBench—61.4%
LiveBench Data Analysis64.3%—
BIG-Bench Hard—82.9%
HellaSwag—89.2%
LiveBench69%—
PIQA—85.9%
WinoGrande—89.2%

Math GPT-4.5 leads

GPT-4.5: 31.0 (#200), Llama 3.1-405B: 17.3 (#283)

BenchmarkGPT-4.5Llama 3.1-405B
OTIS Mock AIME 2024-202537.8%9.7%
LMArena Math14121281
MATH Level 578.6%49.8%
Omni-MATH—24.9%
LiveBench Math69.3%—

Knowledge GPT-4.5 leads

GPT-4.5: 32.4 (#205), Llama 3.1-405B: 30.4 (#221)

BenchmarkGPT-4.5Llama 3.1-405B
GPQA Diamond68.7%50.9%
Confabulations13.6%17.6%
LMArena Expert13941243
Humanity's Last Exam5.4%—
MMLU-Pro—72.3%
GPQA (HELM)—52.2%
ARC (AI2) Challenge—95.3%
MMLU—84.5%
TriviaQA—82.7%

Multimodal Not comparable

GPT-4.5: 37.0 (#68), Llama 3.1-405B: —

BenchmarkGPT-4.5Llama 3.1-405B
LMArena Vision1195—
VPCT45%—

Multilingual GPT-4.5 leads

GPT-4.5: 52.7 (#78), Llama 3.1-405B: 40.9 (#209)

BenchmarkGPT-4.5Llama 3.1-405B
LMArena Non-English14131248
LMArena Chinese14211242
LMArena French14171278
LMArena German14571252
LMArena Japanese14161208
LMArena Korean13921184
LMArena Russian14191265
LMArena Spanish—1259

Instruction Following GPT-4.5 leads

GPT-4.5: 72.8 (#129), Llama 3.1-405B: 66.1 (#209)

BenchmarkGPT-4.5Llama 3.1-405B
LMArena Instruction Following14041259
LiveBench Instruction Following72.3%—
IFEval—81.1%

Long Context GPT-4.5 leads

GPT-4.5: 40.4 (#150), Llama 3.1-405B: 38.4 (#192)

BenchmarkGPT-4.5Llama 3.1-405B
LMArena Longer Query14061266
Fiction.LiveBench63.9%—

Writing & Preference GPT-4.5 leads

GPT-4.5: 57.1 (#129), Llama 3.1-405B: 39.1 (#246)

BenchmarkGPT-4.5Llama 3.1-405B
LMArena Text14171284
LMArena Creative Writing13941262
EQ-Bench Creative Writing1258870
LMArena Multi-Turn14441297
Short-Story Creative Writing75.6%—
WildBench—78.3%
LiveBench Language61.5%—

Frequently asked questions

Is GPT-4.5 better than Llama 3.1-405B?

GPT-4.5 is the stronger model overall, scoring 36.9 to 30.4 on the Noometry Index.

Is GPT-4.5 or Llama 3.1-405B better for coding?

GPT-4.5 scores higher on coding benchmarks: 41.8 versus 32.7 in the Noometry coding category.

How many benchmarks do GPT-4.5 and Llama 3.1-405B share?

26 benchmarks have published results for both models. GPT-4.5 has 42 scored results on Noometry and Llama 3.1-405B has 42.

Related comparisons

Go deeper