Noometry

Model comparison

Claude 3 Haiku vs GPT-4.1 nano

GPT-4.1 nano is the stronger model overall, scoring 27.9 to 25.9 on the Noometry Index.

Last verified . 24 shared benchmarks.

Claude 3 Haiku Anthropic

25.9

Rank #340 Confirmed

GPT-4.1 nano OpenAI

27.9

Rank #327 Confirmed

Summary

  • They share 24 benchmarks with published results for both. Claude 3 Haiku scores higher in 3 categories and GPT-4.1 nano in 6 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in math, where GPT-4.1 nano leads 26.9 to 9.8.
  • The biggest single-benchmark swing is MATH Level 5: 14.9% for Claude 3 Haiku and 70% for GPT-4.1 nano.

Side by side

Claude 3 HaikuGPT-4.1 nano
ProviderAnthropicOpenAI
Noometry Index25.927.9
Released2024-03-072025-04-14
WeightsProprietaryProprietary
Context window—1.05M
Max output—33K
Input $ / M tokens—$0.10
Output $ / M tokens—$0.40
Results tracked3738

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude 3 Haiku leads

Claude 3 Haiku: 26.4 (#325), GPT-4.1 nano: 24.1 (#330)

BenchmarkClaude 3 HaikuGPT-4.1 nano
WeirdML9.8%19%
LMArena Coding11991306
Aider Polyglot—8.9%
SciCode—25.9%
BigCodeBench Instruct39.4%—
BigCodeBench Complete50.1%—
CadEval12%—
HumanEval+68.9%—
MBPP+68.8%—

Agentic & Tool Use Not comparable

Claude 3 Haiku: —, GPT-4.1 nano: 26.5 (#104)

BenchmarkClaude 3 HaikuGPT-4.1 nano
Berkeley Function Calling Leaderboard—33%

Reasoning Claude 3 Haiku leads

Claude 3 Haiku: 16.3 (#307), GPT-4.1 nano: 8.5 (#349)

BenchmarkClaude 3 HaikuGPT-4.1 nano
Kagi LLM Benchmark34.2%33.3%
LMArena Hard Prompts11741286
DTBench50.1%52.5%
LMCA8.8%5.5%
Epoch Capabilities Index118.35129.62
ARC-AGI-2—0%
ARC-AGI-1—0%
CritPt—0%
ForecastBench53.2—
WinoGrande74.2%—

Math GPT-4.1 nano leads

Claude 3 Haiku: 9.8 (#319), GPT-4.1 nano: 26.9 (#252)

BenchmarkClaude 3 HaikuGPT-4.1 nano
OTIS Mock AIME 2024-20251.8%28.9%
LMArena Math11881274
MATH Level 514.9%70%
Omni-MATH—36.7%
FrontierMath (Feb 2025 set)—1%

Knowledge GPT-4.1 nano leads

Claude 3 Haiku: 17.3 (#285), GPT-4.1 nano: 21.8 (#273)

BenchmarkClaude 3 HaikuGPT-4.1 nano
GPQA Diamond36.3%48.9%
LMArena Expert11481272
SimpleQA Verified—6%
MMLU-Pro—55%
Confabulations34.2%—
GPQA (HELM)—50.7%
MMLU73.8%—

Multimodal GPT-4.1 nano leads

Claude 3 Haiku: 23.6 (#128), GPT-4.1 nano: 29.2 (#113)

BenchmarkClaude 3 HaikuGPT-4.1 nano
LMArena Vision9501063
ScienceQA72%—

Multilingual GPT-4.1 nano leads

Claude 3 Haiku: 36.0 (#243), GPT-4.1 nano: 41.6 (#205)

BenchmarkClaude 3 HaikuGPT-4.1 nano
LMArena Non-English11781260
LMArena Chinese11551270
LMArena German11741288
LMArena Japanese11021198
LMArena Russian12041261
LMArena French1195—
LMArena Korean1109—
LMArena Spanish1166—

Instruction Following GPT-4.1 nano leads

Claude 3 Haiku: 61.3 (#247), GPT-4.1 nano: 67.8 (#193)

BenchmarkClaude 3 HaikuGPT-4.1 nano
LMArena Instruction Following11731267
IFEval—84.3%

Long Context Claude 3 Haiku leads

Claude 3 Haiku: 36.1 (#237), GPT-4.1 nano: 23.7 (#296)

BenchmarkClaude 3 HaikuGPT-4.1 nano
LMArena Longer Query11901283
Fiction.LiveBench—25%

Writing & Preference GPT-4.1 nano leads

Claude 3 Haiku: 29.7 (#291), GPT-4.1 nano: 40.5 (#243)

BenchmarkClaude 3 HaikuGPT-4.1 nano
LMArena Text11951285
LMArena Creative Writing11571260
EQ-Bench Creative Writing717946
LMArena Multi-Turn11901277
WildBench—81.2%

Frequently asked questions

Is Claude 3 Haiku better than GPT-4.1 nano?

GPT-4.1 nano is the stronger model overall, scoring 27.9 to 25.9 on the Noometry Index.

Is Claude 3 Haiku or GPT-4.1 nano better for coding?

Claude 3 Haiku scores higher on coding benchmarks: 26.4 versus 24.1 in the Noometry coding category.

How many benchmarks do Claude 3 Haiku and GPT-4.1 nano share?

24 benchmarks have published results for both models. Claude 3 Haiku has 37 scored results on Noometry and GPT-4.1 nano has 38.

Related comparisons

Go deeper