Noometry

Model comparison

Claude 3.5 Haiku vs GPT-4.1

GPT-4.1 is the stronger model overall, scoring 35.9 to 29.2 on the Noometry Index.

Last verified . 34 shared benchmarks.

Claude 3.5 Haiku Anthropic

29.2

Rank #315 Confirmed

GPT-4.1 OpenAI

35.9

Rank #219 Confirmed

Summary

  • They share 34 benchmarks with published results for both. Claude 3.5 Haiku scores higher in 1 category and GPT-4.1 in 9 categories; 10 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where GPT-4.1 leads 37.1 to 18.7.
  • The biggest single-benchmark swing is GeoBench: 34% for Claude 3.5 Haiku and 72% for GPT-4.1.

Side by side

Claude 3.5 HaikuGPT-4.1
ProviderAnthropicOpenAI
Noometry Index29.235.9
Released2024-10-222025-04-14
WeightsProprietaryProprietary
Context window—1.05M
Max output—33K
Input $ / M tokens—$2
Output $ / M tokens—$8
Results tracked4952

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4.1 leads

Claude 3.5 Haiku: 32.9 (#265), GPT-4.1: 34.4 (#238)

BenchmarkClaude 3.5 HaikuGPT-4.1
Aider Polyglot28%52.4%
WeirdML30.7%39%
LMArena Coding12861391
CadEval32%42%
SWE-bench Verified—48.5%
SWE-bench Verified (bash only)—39.6%
SciCode27.4%—
BigCodeBench Instruct46.1%—
LiveBench Coding51.4%—
BigCodeBench Complete59%—
ALE-Bench—558.1

Agentic & Tool Use GPT-4.1 leads

Claude 3.5 Haiku: 28.0 (#95), GPT-4.1: 34.7 (#43)

BenchmarkClaude 3.5 HaikuGPT-4.1
Berkeley Function Calling Leaderboard—54%
BALROG19.3%—

Reasoning Claude 3.5 Haiku leads

Claude 3.5 Haiku: 17.7 (#290), GPT-4.1: 11.7 (#339)

BenchmarkClaude 3.5 HaikuGPT-4.1
LMArena Hard Prompts12511384
DTBench56.7%68.3%
Epoch Capabilities Index127.15136.78
ARC-AGI-2—0.4%
SimpleBench—27%
Kagi LLM Benchmark—52.3%
ARC-AGI-1—5.5%
CritPt0%—
Chess Puzzles—6%
EnigmaEval—2.2%
LiveBench Reasoning28.1%—
LiveBench Data Analysis48.5%—
LMCA—25.6%
ForecastBench—61.5
LiveBench43.5%—

Math GPT-4.1 leads

Claude 3.5 Haiku: 14.7 (#300), GPT-4.1: 22.3 (#280)

BenchmarkClaude 3.5 HaikuGPT-4.1
OTIS Mock AIME 2024-20254.3%38.3%
Omni-MATH22.4%47.1%
LMArena Math12441370
MATH Level 546.4%83%
FrontierMath (Feb 2025 set)0.3%5.5%
FrontierMath (Tiers 1-3)—6%
LiveBench Math35.5%—
FrontierMath Tier 4 (v1)—0%

Knowledge GPT-4.1 leads

Claude 3.5 Haiku: 18.7 (#281), GPT-4.1: 37.1 (#160)

BenchmarkClaude 3.5 HaikuGPT-4.1
GPQA Diamond38.1%66.9%
MMLU-Pro60.5%81.1%
GPQA (HELM)36.3%65.9%
LMArena Expert12081364
Humanity's Last Exam—5.4%
SimpleQA Verified—31.1%
Confabulations36.7%—
Vectara Hallucination Rate—5.6%
MMLU74.3%—

Multimodal GPT-4.1 leads

Claude 3.5 Haiku: 26.8 (#117), GPT-4.1: 38.2 (#67)

BenchmarkClaude 3.5 HaikuGPT-4.1
LMArena Vision10921211
GeoBench34%72%

Multilingual GPT-4.1 leads

Claude 3.5 Haiku: 40.0 (#218), GPT-4.1: 49.4 (#133)

BenchmarkClaude 3.5 HaikuGPT-4.1
LMArena Non-English12381370
LMArena Chinese12291382
LMArena French12641382
LMArena German12371381
LMArena Japanese11751319
LMArena Korean11731339
LMArena Russian12531377
LMArena Spanish12611376

Instruction Following GPT-4.1 leads

Claude 3.5 Haiku: 62.9 (#234), GPT-4.1: 71.3 (#153)

BenchmarkClaude 3.5 HaikuGPT-4.1
IFEval79.2%83.8%
LMArena Instruction Following12411367
LiveBench Instruction Following61.9%—

Long Context GPT-4.1 leads

Claude 3.5 Haiku: 38.3 (#200), GPT-4.1: 40.0 (#163)

BenchmarkClaude 3.5 HaikuGPT-4.1
LMArena Longer Query12611385
Fiction.LiveBench—63.9%

Writing & Preference GPT-4.1 leads

Claude 3.5 Haiku: 42.7 (#234), GPT-4.1: 57.6 (#125)

BenchmarkClaude 3.5 HaikuGPT-4.1
LMArena Text12551383
LMArena Creative Writing12331363
EQ-Bench Creative Writing11461420
WildBench76%85.4%
LMArena Multi-Turn12651398
Short-Story Creative Writing73.5%—
LiveBench Language35.4%—

Frequently asked questions

Is Claude 3.5 Haiku better than GPT-4.1?

GPT-4.1 is the stronger model overall, scoring 35.9 to 29.2 on the Noometry Index.

Is Claude 3.5 Haiku or GPT-4.1 better for coding?

GPT-4.1 scores higher on coding benchmarks: 34.4 versus 32.9 in the Noometry coding category.

How many benchmarks do Claude 3.5 Haiku and GPT-4.1 share?

34 benchmarks have published results for both models. Claude 3.5 Haiku has 49 scored results on Noometry and GPT-4.1 has 52.

Related comparisons

Go deeper