Noometry

Model comparison

Claude 3.5 Haiku vs Llama-3.3-70B-Instruct

Llama-3.3-70B-Instruct is the stronger model overall, scoring 30.6 to 29.2 on the Noometry Index.

Last verified . 37 shared benchmarks.

Claude 3.5 Haiku Anthropic

29.2

Rank #315 Confirmed

Llama-3.3-70B-Instruct Meta

30.6

Rank #291 Confirmed

Summary

  • They share 37 benchmarks with published results for both. Claude 3.5 Haiku scores higher in 5 categories and Llama-3.3-70B-Instruct in 4 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Llama-3.3-70B-Instruct leads 30.6 to 18.7.
  • The biggest single-benchmark swing is LiveBench Reasoning: 28.1% for Claude 3.5 Haiku and 50.8% for Llama-3.3-70B-Instruct.
  • Llama-3.3-70B-Instruct has downloadable open weights; the other is API-only.

Side by side

Claude 3.5 HaikuLlama-3.3-70B-Instruct
ProviderAnthropicMeta
Noometry Index29.230.6
Released2024-10-222024-12-06
WeightsProprietaryOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.10
Output $ / M tokens—$0.32
Results tracked4943

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude 3.5 Haiku leads

Claude 3.5 Haiku: 32.9 (#265), Llama-3.3-70B-Instruct: 31.0 (#290)

BenchmarkClaude 3.5 HaikuLlama-3.3-70B-Instruct
SciCode27.4%26%
WeirdML30.7%14.4%
BigCodeBench Instruct46.1%46.9%
LiveBench Coding51.4%36.6%
LMArena Coding12861268
BigCodeBench Complete59%57.5%
Aider Polyglot28%—
CadEval32%—

Agentic & Tool Use Claude 3.5 Haiku leads

Claude 3.5 Haiku: 28.0 (#95), Llama-3.3-70B-Instruct: 25.8 (#105)

BenchmarkClaude 3.5 HaikuLlama-3.3-70B-Instruct
BALROG19.3%23%
Berkeley Function Calling Leaderboard—31.9%

Reasoning Claude 3.5 Haiku leads

Claude 3.5 Haiku: 17.7 (#290), Llama-3.3-70B-Instruct: 14.1 (#327)

BenchmarkClaude 3.5 HaikuLlama-3.3-70B-Instruct
CritPt0%0%
LiveBench Reasoning28.1%50.8%
LMArena Hard Prompts12511257
DTBench56.7%59.5%
LiveBench Data Analysis48.5%49.5%
Epoch Capabilities Index127.15127.33
LiveBench43.5%50.2%
SimpleBench—19.9%
LMCA—17.5%
ForecastBench—58.6

Math Too close to call

Claude 3.5 Haiku: 14.7 (#300), Llama-3.3-70B-Instruct: 15.3 (#298)

BenchmarkClaude 3.5 HaikuLlama-3.3-70B-Instruct
OTIS Mock AIME 2024-20254.3%5.1%
LiveBench Math35.5%42.2%
LMArena Math12441267
MATH Level 546.4%41.6%
Omni-MATH22.4%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge Llama-3.3-70B-Instruct leads

Claude 3.5 Haiku: 18.7 (#281), Llama-3.3-70B-Instruct: 30.6 (#226)

BenchmarkClaude 3.5 HaikuLlama-3.3-70B-Instruct
GPQA Diamond38.1%47.4%
Confabulations36.7%22.8%
LMArena Expert12081225
MMLU74.3%86.3%
MMLU-Pro60.5%—
Vectara Hallucination Rate—4.1%
GPQA (HELM)36.3%—

Multimodal Not comparable

Claude 3.5 Haiku: 26.8 (#117), Llama-3.3-70B-Instruct: —

BenchmarkClaude 3.5 HaikuLlama-3.3-70B-Instruct
LMArena Vision1092—
GeoBench34%—

Multilingual Too close to call

Claude 3.5 Haiku: 40.0 (#218), Llama-3.3-70B-Instruct: 39.9 (#220)

BenchmarkClaude 3.5 HaikuLlama-3.3-70B-Instruct
LMArena Non-English12381236
LMArena Chinese12291217
LMArena French12641281
LMArena German12371251
LMArena Japanese11751150
LMArena Korean11731143
LMArena Russian12531252
LMArena Spanish12611270

Instruction Following Llama-3.3-70B-Instruct leads

Claude 3.5 Haiku: 62.9 (#234), Llama-3.3-70B-Instruct: 71.1 (#157)

BenchmarkClaude 3.5 HaikuLlama-3.3-70B-Instruct
LiveBench Instruction Following61.9%82.7%
LMArena Instruction Following12411242
IFEval79.2%—

Long Context Claude 3.5 Haiku leads

Claude 3.5 Haiku: 38.3 (#200), Llama-3.3-70B-Instruct: 26.4 (#295)

BenchmarkClaude 3.5 HaikuLlama-3.3-70B-Instruct
LMArena Longer Query12611256
Fiction.LiveBench—33.3%

Writing & Preference Llama-3.3-70B-Instruct leads

Claude 3.5 Haiku: 42.7 (#234), Llama-3.3-70B-Instruct: 47.6 (#207)

BenchmarkClaude 3.5 HaikuLlama-3.3-70B-Instruct
LMArena Text12551274
LMArena Creative Writing12331250
LMArena Multi-Turn12651280
LiveBench Language35.4%39.2%
Short-Story Creative Writing73.5%—
EQ-Bench Creative Writing1146—
WildBench76%—

Frequently asked questions

Is Claude 3.5 Haiku better than Llama-3.3-70B-Instruct?

Llama-3.3-70B-Instruct is the stronger model overall, scoring 30.6 to 29.2 on the Noometry Index.

Is Claude 3.5 Haiku or Llama-3.3-70B-Instruct better for coding?

Claude 3.5 Haiku scores higher on coding benchmarks: 32.9 versus 31.0 in the Noometry coding category.

How many benchmarks do Claude 3.5 Haiku and Llama-3.3-70B-Instruct share?

37 benchmarks have published results for both models. Claude 3.5 Haiku has 49 scored results on Noometry and Llama-3.3-70B-Instruct has 43.

Related comparisons

Go deeper