Noometry

Model comparison

Claude 3 Haiku vs Llama 4 Scout

Llama 4 Scout is the stronger model overall, scoring 27.7 to 25.9 on the Noometry Index.

Last verified . 28 shared benchmarks.

Claude 3 Haiku Anthropic

25.9

Rank #340 Confirmed

Llama 4 Scout Meta

27.7

Rank #330 Confirmed

Summary

  • They share 28 benchmarks with published results for both. Claude 3 Haiku scores higher in 3 categories and Llama 4 Scout in 6 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Llama 4 Scout leads 31.9 to 17.3.
  • The biggest single-benchmark swing is MATH Level 5: 14.9% for Claude 3 Haiku and 62.3% for Llama 4 Scout.
  • Llama 4 Scout has downloadable open weights; the other is API-only.

Side by side

Claude 3 HaikuLlama 4 Scout
ProviderAnthropicMeta
Noometry Index25.927.7
Released2024-03-072025-04-05
WeightsProprietaryOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.10
Output $ / M tokens—$0.30
Results tracked3743

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude 3 Haiku leads

Claude 3 Haiku: 26.4 (#325), Llama 4 Scout: 20.2 (#339)

BenchmarkClaude 3 HaikuLlama 4 Scout
LMArena Coding11991286
BigCodeBench Complete50.1%43.1%
SWE-bench Verified (bash only)—9.1%
SciCode—17%
WeirdML9.8%—
BigCodeBench Instruct39.4%—
CadEval12%—
HumanEval+68.9%—
MBPP+68.8%—

Agentic & Tool Use Not comparable

Claude 3 Haiku: —, Llama 4 Scout: 24.6 (#119)

BenchmarkClaude 3 HaikuLlama 4 Scout
Berkeley Function Calling Leaderboard—28.1%

Reasoning Claude 3 Haiku leads

Claude 3 Haiku: 16.3 (#307), Llama 4 Scout: 9.1 (#345)

BenchmarkClaude 3 HaikuLlama 4 Scout
Kagi LLM Benchmark34.2%36.9%
LMArena Hard Prompts11741266
DTBench50.1%57.9%
LMCA8.8%12%
Epoch Capabilities Index118.35129.64
ForecastBench53.257.5
ARC-AGI-2—0%
ARC-AGI-1—0.5%
CritPt—0%
WinoGrande74.2%—

Math Llama 4 Scout leads

Claude 3 Haiku: 9.8 (#319), Llama 4 Scout: 19.6 (#286)

BenchmarkClaude 3 HaikuLlama 4 Scout
OTIS Mock AIME 2024-20251.8%7.8%
LMArena Math11881287
MATH Level 514.9%62.3%
Omni-MATH—37.3%
FrontierMath (Feb 2025 set)—0%

Knowledge Llama 4 Scout leads

Claude 3 Haiku: 17.3 (#285), Llama 4 Scout: 31.9 (#217)

BenchmarkClaude 3 HaikuLlama 4 Scout
GPQA Diamond36.3%51.8%
LMArena Expert11481235
MMLU-Pro—74.2%
Confabulations34.2%—
Vectara Hallucination Rate—7.7%
GPQA (HELM)—50.7%
MMLU73.8%—

Multimodal Llama 4 Scout leads

Claude 3 Haiku: 23.6 (#128), Llama 4 Scout: 32.2 (#102)

BenchmarkClaude 3 HaikuLlama 4 Scout
LMArena Vision9501118
ScienceQA72%—
SpatialViz-Bench—34.2%

Multilingual Llama 4 Scout leads

Claude 3 Haiku: 36.0 (#243), Llama 4 Scout: 41.0 (#212)

BenchmarkClaude 3 HaikuLlama 4 Scout
LMArena Non-English11781252
LMArena Chinese11551255
LMArena French11951282
LMArena German11741272
LMArena Japanese11021206
LMArena Korean11091207
LMArena Russian12041263
LMArena Spanish11661278

Instruction Following Llama 4 Scout leads

Claude 3 Haiku: 61.3 (#247), Llama 4 Scout: 65.8 (#217)

BenchmarkClaude 3 HaikuLlama 4 Scout
LMArena Instruction Following11731248
IFEval—81.8%

Long Context Claude 3 Haiku leads

Claude 3 Haiku: 36.1 (#237), Llama 4 Scout: 27.5 (#294)

BenchmarkClaude 3 HaikuLlama 4 Scout
LMArena Longer Query11901265
Fiction.LiveBench—36%

Writing & Preference Llama 4 Scout leads

Claude 3 Haiku: 29.7 (#291), Llama 4 Scout: 37.0 (#261)

BenchmarkClaude 3 HaikuLlama 4 Scout
LMArena Text11951279
LMArena Creative Writing11571249
EQ-Bench Creative Writing717783
LMArena Multi-Turn11901280
WildBench—78%

Frequently asked questions

Is Claude 3 Haiku better than Llama 4 Scout?

Llama 4 Scout is the stronger model overall, scoring 27.7 to 25.9 on the Noometry Index.

Is Claude 3 Haiku or Llama 4 Scout better for coding?

Claude 3 Haiku scores higher on coding benchmarks: 26.4 versus 20.2 in the Noometry coding category.

How many benchmarks do Claude 3 Haiku and Llama 4 Scout share?

28 benchmarks have published results for both models. Claude 3 Haiku has 37 scored results on Noometry and Llama 4 Scout has 43.

Related comparisons

Go deeper