Noometry

Model comparison

Claude 3 Haiku vs GPT-4 Turbo

GPT-4 Turbo is the stronger model overall, scoring 30.5 to 25.9 on the Noometry Index.

Last verified . 32 shared benchmarks.

Claude 3 Haiku Anthropic

25.9

Rank #340 Confirmed

GPT-4 Turbo OpenAI

30.5

Rank #292 Confirmed

Summary

  • They share 32 benchmarks with published results for both. Claude 3 Haiku scores higher in 2 categories and GPT-4 Turbo in 7 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where GPT-4 Turbo leads 47.7 to 29.7.
  • The biggest single-benchmark swing is MATH Level 5: 14.9% for Claude 3 Haiku and 46.7% for GPT-4 Turbo.

Side by side

Claude 3 HaikuGPT-4 Turbo
ProviderAnthropicOpenAI
Noometry Index25.930.5
Released2024-03-072023-11-06
WeightsProprietaryProprietary
Context window—128K
Max output—4K
Input $ / M tokens—$10
Output $ / M tokens—$30
Results tracked3736

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4 Turbo leads

Claude 3 Haiku: 26.4 (#325), GPT-4 Turbo: 33.8 (#249)

BenchmarkClaude 3 HaikuGPT-4 Turbo
WeirdML9.8%18%
BigCodeBench Instruct39.4%48.2%
LMArena Coding11991268
BigCodeBench Complete50.1%58.2%
HumanEval+68.9%86.6%
MBPP+68.8%73.3%
CadEval12%—

Agentic & Tool Use Not comparable

Claude 3 Haiku: —, GPT-4 Turbo: —

BenchmarkClaude 3 HaikuGPT-4 Turbo
METR Time Horizons—36.7%

Reasoning Too close to call

Claude 3 Haiku: 16.3 (#307), GPT-4 Turbo: 15.3 (#317)

BenchmarkClaude 3 HaikuGPT-4 Turbo
LMArena Hard Prompts11741251
DTBench50.1%61.6%
LMCA8.8%9.8%
Epoch Capabilities Index118.35127.25
ForecastBench53.259.4
SimpleBench—25.1%
Kagi LLM Benchmark34.2%—
Chess Puzzles—6%
WinoGrande74.2%—

Math Too close to call

Claude 3 Haiku: 9.8 (#319), GPT-4 Turbo: 9.0 (#322)

BenchmarkClaude 3 HaikuGPT-4 Turbo
OTIS Mock AIME 2024-20251.8%6.7%
LMArena Math11881272
MATH Level 514.9%46.7%
FrontierMath (Tiers 1-3)—0.7%

Knowledge GPT-4 Turbo leads

Claude 3 Haiku: 17.3 (#285), GPT-4 Turbo: 24.3 (#268)

BenchmarkClaude 3 HaikuGPT-4 Turbo
GPQA Diamond36.3%46.6%
Confabulations34.2%28.4%
LMArena Expert11481223
MMLU73.8%81.3%

Multimodal GPT-4 Turbo leads

Claude 3 Haiku: 23.6 (#128), GPT-4 Turbo: 30.6 (#110)

BenchmarkClaude 3 HaikuGPT-4 Turbo
LMArena Vision9501090
ScienceQA72%—

Multilingual GPT-4 Turbo leads

Claude 3 Haiku: 36.0 (#243), GPT-4 Turbo: 40.5 (#216)

BenchmarkClaude 3 HaikuGPT-4 Turbo
LMArena Non-English11781245
LMArena Chinese11551242
LMArena French11951276
LMArena German11741259
LMArena Japanese11021194
LMArena Korean11091187
LMArena Russian12041259
LMArena Spanish11661260

Instruction Following GPT-4 Turbo leads

Claude 3 Haiku: 61.3 (#247), GPT-4 Turbo: 65.8 (#216)

BenchmarkClaude 3 HaikuGPT-4 Turbo
LMArena Instruction Following11731249

Long Context GPT-4 Turbo leads

Claude 3 Haiku: 36.1 (#237), GPT-4 Turbo: 38.0 (#206)

BenchmarkClaude 3 HaikuGPT-4 Turbo
LMArena Longer Query11901254

Writing & Preference GPT-4 Turbo leads

Claude 3 Haiku: 29.7 (#291), GPT-4 Turbo: 47.7 (#206)

BenchmarkClaude 3 HaikuGPT-4 Turbo
LMArena Text11951272
LMArena Creative Writing11571269
LMArena Multi-Turn11901267
EQ-Bench Creative Writing717—

Frequently asked questions

Is Claude 3 Haiku better than GPT-4 Turbo?

GPT-4 Turbo is the stronger model overall, scoring 30.5 to 25.9 on the Noometry Index.

Is Claude 3 Haiku or GPT-4 Turbo better for coding?

GPT-4 Turbo scores higher on coding benchmarks: 33.8 versus 26.4 in the Noometry coding category.

How many benchmarks do Claude 3 Haiku and GPT-4 Turbo share?

32 benchmarks have published results for both models. Claude 3 Haiku has 37 scored results on Noometry and GPT-4 Turbo has 36.

Related comparisons

Go deeper