Noometry

Model comparison

Claude 3 Haiku vs GPT-5.4

GPT-5.4 is the stronger model overall, scoring 59.4 to 25.9 on the Noometry Index.

Last verified . 27 shared benchmarks.

Claude 3 Haiku Anthropic

25.9

Rank #340 Confirmed

GPT-5.4 OpenAI

59.4

Rank #16 Confirmed

Summary

  • They share 27 benchmarks with published results for both. Claude 3 Haiku scores higher in 0 categories and GPT-5.4 in 9 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in math, where GPT-5.4 leads 73.5 to 9.8.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 1.8% for Claude 3 Haiku and 97.8% for GPT-5.4.

Side by side

Claude 3 HaikuGPT-5.4
ProviderAnthropicOpenAI
Noometry Index25.959.4
Released2024-03-072026-03-05
WeightsProprietaryProprietary
Context window—1.05M
Max output—128K
Input $ / M tokens—$2.50
Output $ / M tokens—$15
Results tracked3768

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.4 leads

Claude 3 Haiku: 26.4 (#325), GPT-5.4: 52.6 (#33)

BenchmarkClaude 3 HaikuGPT-5.4
WeirdML9.8%77.7%
LMArena Coding11991497
SWE-bench Verified—76.9%
DeepSWE—51.8%
LMArena WebDev—1465
SciCode—56.6%
GSO—31.4%
BigCodeBench Instruct39.4%—
MirrorCode—15.6%
BigCodeBench Complete50.1%—
CadEval12%—
ALE-Bench—1,607
AlgoTune—1.85
HumanEval+68.9%—
MBPP+68.8%—

Agentic & Tool Use Not comparable

Claude 3 Haiku: —, GPT-5.4: 46.5 (#13)

BenchmarkClaude 3 HaikuGPT-5.4
Terminal-Bench—81.8%
APEX-Agents—52.4%
τ²-bench Banking—39.4%
DeepResearch Bench—35.1%
PostTrainBench—19%
GBAEval—45.1%
LMArena Search—1197
METR Time Horizons—74.3%
Vending-Bench 2—6,144

Reasoning GPT-5.4 leads

Claude 3 Haiku: 16.3 (#307), GPT-5.4: 61.8 (#19)

BenchmarkClaude 3 HaikuGPT-5.4
Kagi LLM Benchmark34.2%63.8%
LMArena Hard Prompts11741485
DTBench50.1%94.4%
LMCA8.8%52%
Epoch Capabilities Index118.35156.81
ForecastBench53.259.5
ARC-AGI-2—74%
NYT Connections (extended)—91.3%
ARC-AGI-1—93.7%
CritPt—23.4%
Chess Puzzles—44%
EnigmaEval—16%
Thematic Generalization—80%
EBR-Bench—25.4%
Mystery Game Puzzles—37%
WinoGrande74.2%—

Math GPT-5.4 leads

Claude 3 Haiku: 9.8 (#319), GPT-5.4: 73.5 (#19)

BenchmarkClaude 3 HaikuGPT-5.4
OTIS Mock AIME 2024-20251.8%97.8%
LMArena Math11881488
FrontierMath (Tiers 1-3)—78.6%
FrontierMath Tier 4—49%
MathArena Final-Answer Competitions—83.1%
ProofBench—56%
MATH Level 514.9%—
FrontierMath (Feb 2025 set)—47.6%
FrontierMath Tier 4 (v1)—27.1%

Knowledge GPT-5.4 leads

Claude 3 Haiku: 17.3 (#285), GPT-5.4: 65.3 (#14)

BenchmarkClaude 3 HaikuGPT-5.4
GPQA Diamond36.3%93.3%
LMArena Expert11481507
Humanity's Last Exam—36.2%
SimpleQA Verified—45.1%
Confabulations34.2%—
Vectara Hallucination Rate—7%
MMLU73.8%—

Multimodal GPT-5.4 leads

Claude 3 Haiku: 23.6 (#128), GPT-5.4: 43.7 (#20)

BenchmarkClaude 3 HaikuGPT-5.4
LMArena Vision9501303
Blueprint-Bench 2—27.1%
Furniture Assembly—37.5%
LMArena Document—1471
ScienceQA72%—

Multilingual GPT-5.4 leads

Claude 3 Haiku: 36.0 (#243), GPT-5.4: 56.2 (#23)

BenchmarkClaude 3 HaikuGPT-5.4
LMArena Non-English11781465
LMArena Chinese11551519
LMArena French11951493
LMArena German11741472
LMArena Japanese11021485
LMArena Korean11091448
LMArena Russian12041480
LMArena Spanish11661454

Instruction Following GPT-5.4 leads

Claude 3 Haiku: 61.3 (#247), GPT-5.4: 77.1 (#27)

BenchmarkClaude 3 HaikuGPT-5.4
LMArena Instruction Following11731469

Long Context GPT-5.4 leads

Claude 3 Haiku: 36.1 (#237), GPT-5.4: 50.3 (#8)

BenchmarkClaude 3 HaikuGPT-5.4
LMArena Longer Query11901473
CL-bench—27.9%
CL-bench Life—21.7%

Writing & Preference GPT-5.4 leads

Claude 3 Haiku: 29.7 (#291), GPT-5.4: 71.9 (#17)

BenchmarkClaude 3 HaikuGPT-5.4
LMArena Text11951469
LMArena Creative Writing11571439
EQ-Bench Creative Writing7171840
LMArena Multi-Turn11901482
EQ-Bench 4—1272

Frequently asked questions

Is Claude 3 Haiku better than GPT-5.4?

GPT-5.4 is the stronger model overall, scoring 59.4 to 25.9 on the Noometry Index.

Is Claude 3 Haiku or GPT-5.4 better for coding?

GPT-5.4 scores higher on coding benchmarks: 52.6 versus 26.4 in the Noometry coding category.

How many benchmarks do Claude 3 Haiku and GPT-5.4 share?

27 benchmarks have published results for both models. Claude 3 Haiku has 37 scored results on Noometry and GPT-5.4 has 68.

Related comparisons

Go deeper