Noometry

Model comparison

Claude 3 Haiku vs GPT-4.1 mini

GPT-4.1 mini is the stronger model overall, scoring 33.5 to 25.7 on the Noometry Index.

Last verified . 29 shared benchmarks.

Claude 3 Haiku Anthropic

25.7

Rank #333 Confirmed

GPT-4.1 mini OpenAI

33.5

Rank #232 Confirmed

Summary

  • They share 29 benchmarks with published results for both. Claude 3 Haiku scores higher in 2 categories and GPT-4.1 mini in 7 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where GPT-4.1 mini leads 48.8 to 29.8.
  • The biggest single-benchmark swing is MATH Level 5: 14.9% for Claude 3 Haiku and 87.3% for GPT-4.1 mini.

Side by side

Claude 3 HaikuGPT-4.1 mini
ProviderAnthropicOpenAI
Noometry Index25.733.5
Released2024-03-072025-04-14
WeightsProprietaryProprietary
Context window—1.05M
Max output—33K
Input $ / M tokens—$0.40
Output $ / M tokens—$1.60
Results tracked3747

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4.1 mini leads

Claude 3 Haiku: 26.1 (#320), GPT-4.1 mini: 30.4 (#286)

BenchmarkClaude 3 HaikuGPT-4.1 mini
WeirdML9.8%37.6%
BigCodeBench Instruct39.4%48.9%
LMArena Coding11991367
CadEval12%16%
SWE-bench Verified (bash only)—23.9%
Aider Polyglot—32.4%
SciCode—40.4%
BigCodeBench Complete50.1%—
HumanEval+68.9%—
MBPP+68.8%—

Agentic & Tool Use Not comparable

Claude 3 Haiku: —, GPT-4.1 mini: 33.0 (#52)

BenchmarkClaude 3 HaikuGPT-4.1 mini
Berkeley Function Calling Leaderboard—50.5%

Reasoning Claude 3 Haiku leads

Claude 3 Haiku: 15.7 (#303), GPT-4.1 mini: 10.9 (#335)

BenchmarkClaude 3 HaikuGPT-4.1 mini
Kagi LLM Benchmark34.2%48.6%
LMArena Hard Prompts11741349
DTBench50.1%68.8%
LMCA8.8%21.1%
Epoch Capabilities Index118.3135.01
ARC-AGI-2—0%
ARC-AGI-1—3.5%
CritPt—0%
Chess Puzzles—7%
Mystery Game Puzzles—7%
ForecastBench53.2—
WinoGrande74.2%—

Math GPT-4.1 mini leads

Claude 3 Haiku: 9.2 (#313), GPT-4.1 mini: 23.1 (#264)

BenchmarkClaude 3 HaikuGPT-4.1 mini
OTIS Mock AIME 2024-20251.8%44.7%
LMArena Math11881343
MATH Level 514.9%87.3%
FrontierMath (Tiers 1-3)—6.7%
Omni-MATH—49.1%
FrontierMath (Feb 2025 set)—4.5%

Knowledge GPT-4.1 mini leads

Claude 3 Haiku: 17.3 (#280), GPT-4.1 mini: 34.7 (#183)

BenchmarkClaude 3 HaikuGPT-4.1 mini
GPQA Diamond36.3%65.8%
LMArena Expert11481338
SimpleQA Verified—12.7%
MMLU-Pro—78.3%
Confabulations34.2%—
GPQA (HELM)—61.4%
MMLU73.8%—

Multimodal GPT-4.1 mini leads

Claude 3 Haiku: 23.1 (#123), GPT-4.1 mini: 35.2 (#77)

BenchmarkClaude 3 HaikuGPT-4.1 mini
LMArena Vision9501181
ScienceQA72%—

Multilingual GPT-4.1 mini leads

Claude 3 Haiku: 36.1 (#238), GPT-4.1 mini: 45.9 (#161)

BenchmarkClaude 3 HaikuGPT-4.1 mini
LMArena Non-English11781318
LMArena Chinese11551329
LMArena French11941357
LMArena German11741349
LMArena Japanese11021291
LMArena Korean11081298
LMArena Russian12041324
LMArena Spanish11651319

Instruction Following GPT-4.1 mini leads

Claude 3 Haiku: 61.5 (#242), GPT-4.1 mini: 73.9 (#113)

BenchmarkClaude 3 HaikuGPT-4.1 mini
LMArena Instruction Following11741332
IFEval—90.4%

Long Context Claude 3 Haiku leads

Claude 3 Haiku: 36.1 (#232), GPT-4.1 mini: 31.8 (#270)

BenchmarkClaude 3 HaikuGPT-4.1 mini
LMArena Longer Query11911344
Fiction.LiveBench—44.4%

Writing & Preference GPT-4.1 mini leads

Claude 3 Haiku: 29.8 (#286), GPT-4.1 mini: 48.8 (#193)

BenchmarkClaude 3 HaikuGPT-4.1 mini
LMArena Text11951341
LMArena Creative Writing11571300
EQ-Bench Creative Writing7171147
LMArena Multi-Turn11901354
WildBench—83.8%

Frequently asked questions

Is Claude 3 Haiku better than GPT-4.1 mini?

GPT-4.1 mini is the stronger model overall, scoring 33.5 to 25.7 on the Noometry Index.

Is Claude 3 Haiku or GPT-4.1 mini better for coding?

GPT-4.1 mini scores higher on coding benchmarks: 30.4 versus 26.1 in the Noometry coding category.

How many benchmarks do Claude 3 Haiku and GPT-4.1 mini share?

29 benchmarks have published results for both models. Claude 3 Haiku has 37 scored results on Noometry and GPT-4.1 mini has 47.

Related comparisons

Go deeper