Noometry

Model comparison

Claude 3 Haiku vs Claude 3 Opus

Claude 3 Opus is the stronger model overall, scoring 29.5 to 25.9 on the Noometry Index.

Last verified . 33 shared benchmarks.

Claude 3 Haiku Anthropic

25.9

Rank #340 Confirmed

Claude 3 Opus Anthropic

29.5

Rank #310 Confirmed

Summary

  • They share 33 benchmarks with published results for both. Claude 3 Haiku scores higher in 1 category and Claude 3 Opus in 8 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Claude 3 Opus leads 47.2 to 29.7.
  • The biggest single-benchmark swing is MATH Level 5: 14.9% for Claude 3 Haiku and 37.5% for Claude 3 Opus.

Side by side

Claude 3 HaikuClaude 3 Opus
ProviderAnthropicAnthropic
Noometry Index25.929.5
Released2024-03-072024-02-29
WeightsProprietaryProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3746

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude 3 Opus leads

Claude 3 Haiku: 26.4 (#325), Claude 3 Opus: 32.9 (#267)

BenchmarkClaude 3 HaikuClaude 3 Opus
WeirdML9.8%19.2%
BigCodeBench Instruct39.4%45.5%
LMArena Coding11991264
BigCodeBench Complete50.1%57.4%
HumanEval+68.9%77.4%
MBPP+68.8%73.3%
LiveBench Coding—38.6%
CadEval12%—

Agentic & Tool Use Not comparable

Claude 3 Haiku: —, Claude 3 Opus: 24.6 (#116)

BenchmarkClaude 3 HaikuClaude 3 Opus
Cybench—10%
METR Time Horizons—29.5%

Reasoning Claude 3 Haiku leads

Claude 3 Haiku: 16.3 (#307), Claude 3 Opus: 14.6 (#324)

BenchmarkClaude 3 HaikuClaude 3 Opus
LMArena Hard Prompts11741245
DTBench50.1%61.6%
LMCA8.8%17%
Epoch Capabilities Index118.35126.91
ForecastBench53.258.4
WinoGrande74.2%88.5%
SimpleBench—23.5%
Kagi LLM Benchmark34.2%—
Chess Puzzles—5%
EnigmaEval—0.8%
LiveBench Reasoning—40.6%
LiveBench Data Analysis—57.9%
LiveBench—49.2%

Math Claude 3 Opus leads

Claude 3 Haiku: 9.8 (#319), Claude 3 Opus: 14.8 (#299)

BenchmarkClaude 3 HaikuClaude 3 Opus
OTIS Mock AIME 2024-20251.8%4.7%
LMArena Math11881273
MATH Level 514.9%37.5%
LiveBench Math—43.6%

Knowledge Claude 3 Opus leads

Claude 3 Haiku: 17.3 (#285), Claude 3 Opus: 24.5 (#267)

BenchmarkClaude 3 HaikuClaude 3 Opus
GPQA Diamond36.3%47.2%
Confabulations34.2%22.7%
LMArena Expert11481223
MMLU73.8%84.6%
SimpleQA Verified—12.6%

Multimodal Claude 3 Opus leads

Claude 3 Haiku: 23.6 (#128), Claude 3 Opus: 27.1 (#116)

BenchmarkClaude 3 HaikuClaude 3 Opus
LMArena Vision9501023
ScienceQA72%—

Multilingual Claude 3 Opus leads

Claude 3 Haiku: 36.0 (#243), Claude 3 Opus: 41.4 (#207)

BenchmarkClaude 3 HaikuClaude 3 Opus
LMArena Non-English11781258
LMArena Chinese11551248
LMArena French11951275
LMArena German11741258
LMArena Japanese11021204
LMArena Korean11091187
LMArena Russian12041280
LMArena Spanish11661246

Instruction Following Claude 3 Opus leads

Claude 3 Haiku: 61.3 (#247), Claude 3 Opus: 64.1 (#228)

BenchmarkClaude 3 HaikuClaude 3 Opus
LMArena Instruction Following11731248
LiveBench Instruction Following—63.9%

Long Context Claude 3 Opus leads

Claude 3 Haiku: 36.1 (#237), Claude 3 Opus: 38.2 (#202)

BenchmarkClaude 3 HaikuClaude 3 Opus
LMArena Longer Query11901259

Writing & Preference Claude 3 Opus leads

Claude 3 Haiku: 29.7 (#291), Claude 3 Opus: 47.2 (#213)

BenchmarkClaude 3 HaikuClaude 3 Opus
LMArena Text11951262
LMArena Creative Writing11571235
LMArena Multi-Turn11901275
EQ-Bench Creative Writing717—
LiveBench Language—50.4%

Frequently asked questions

Is Claude 3 Haiku better than Claude 3 Opus?

Claude 3 Opus is the stronger model overall, scoring 29.5 to 25.9 on the Noometry Index.

Is Claude 3 Haiku or Claude 3 Opus better for coding?

Claude 3 Opus scores higher on coding benchmarks: 32.9 versus 26.4 in the Noometry coding category.

How many benchmarks do Claude 3 Haiku and Claude 3 Opus share?

33 benchmarks have published results for both models. Claude 3 Haiku has 37 scored results on Noometry and Claude 3 Opus has 46.

Related comparisons

Go deeper