Noometry

Model comparison

Claude Haiku 4.5 vs Llama 3.1-70B

Claude Haiku 4.5 is the stronger model overall, scoring 39.5 to 29.6 on the Noometry Index. Llama 3.1-70B costs 5.0× less per token, which makes it the better buy when Claude Haiku 4.5's lead doesn't matter for your workload.

Last verified . 30 shared benchmarks.

Claude Haiku 4.5 Anthropic

39.5

Rank #165 Confirmed

Llama 3.1-70B Meta

29.6

Rank #308 Confirmed

Summary

  • They share 30 benchmarks with published results for both. Claude Haiku 4.5 scores higher in 8 categories and Llama 3.1-70B in 1 category; 9 gaps are clear of the uncertainty.
  • The widest gap is in math, where Claude Haiku 4.5 leads 44.9 to 13.5.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 66.7% for Claude Haiku 4.5 and 3.6% for Llama 3.1-70B.
  • Llama 3.1-70B is cheaper at $0.40 / $0.40 per million input/output tokens, against $1 / $5 for Claude Haiku 4.5.
  • Claude Haiku 4.5 accepts more context: 200K tokens versus 128K.
  • Llama 3.1-70B has downloadable open weights; the other is API-only.

Side by side

Claude Haiku 4.5Llama 3.1-70B
ProviderAnthropicMeta
Noometry Index39.529.6
Released2025-10-152024-07-23
WeightsProprietaryOpen
Context window200K128K
Max output64K4K
Input $ / M tokens$1$0.40
Output $ / M tokens$5$0.40
Results tracked5335

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude Haiku 4.5 leads

Claude Haiku 4.5: 44.0 (#78), Llama 3.1-70B: 30.3 (#296)

BenchmarkClaude Haiku 4.5Llama 3.1-70B
WeirdML45.4%9%
LMArena Coding14531260
SWE-bench Verified (bash only)66.6%—
LMArena WebDev1330—
SWE-bench Multilingual64.7%—
SciCode43.3%—
BigCodeBench Instruct—46.1%
BigCodeBench Complete—54.8%
ALE-Bench653.48—

Agentic & Tool Use Claude Haiku 4.5 leads

Claude Haiku 4.5: 33.6 (#52), Llama 3.1-70B: 25.1 (#112)

BenchmarkClaude Haiku 4.5Llama 3.1-70B
BALROG31.2%27.9%
Terminal-Bench35.5%—
Berkeley Function Calling Leaderboard68.7%—
TheAgentCompany—6.9%
DeepResearch Bench45.5%—
ExploitBench13.7%—
Vending-Bench 2458.89—

Reasoning Llama 3.1-70B leads

Claude Haiku 4.5: 15.1 (#320), Llama 3.1-70B: 21.6 (#220)

BenchmarkClaude Haiku 4.5Llama 3.1-70B
LMArena Hard Prompts14201241
DTBench73.6%60%
LMCA30.9%14.8%
Epoch Capabilities Index142.41125.92
ARC-AGI-24%—
NYT Connections (extended)14.3%—
ARC-AGI-147.7%—
CritPt0%—
Chess Puzzles8%—
ForecastBench61.4—

Math Claude Haiku 4.5 leads

Claude Haiku 4.5: 44.9 (#78), Llama 3.1-70B: 13.5 (#304)

BenchmarkClaude Haiku 4.5Llama 3.1-70B
OTIS Mock AIME 2024-202566.7%3.6%
Omni-MATH56.1%21%
LMArena Math13961252
MATH Level 596.4%36.7%
FrontierMath (Feb 2025 set)5.9%—
FrontierMath Tier 4 (v1)2.1%—

Knowledge Claude Haiku 4.5 leads

Claude Haiku 4.5: 37.7 (#153), Llama 3.1-70B: 24.2 (#269)

BenchmarkClaude Haiku 4.5Llama 3.1-70B
GPQA Diamond71.2%44.2%
MMLU-Pro77.7%65.3%
GPQA (HELM)60.5%42.6%
LMArena Expert14421209
SimpleQA Verified13.2%—
Vectara Hallucination Rate9.8%—
MMLU—80.1%

Multimodal Not comparable

Claude Haiku 4.5: 26.8 (#118), Llama 3.1-70B: —

BenchmarkClaude Haiku 4.5Llama 3.1-70B
Blueprint-Bench 20%—
LMArena Document1420—

Multilingual Claude Haiku 4.5 leads

Claude Haiku 4.5: 49.9 (#129), Llama 3.1-70B: 38.8 (#225)

BenchmarkClaude Haiku 4.5Llama 3.1-70B
LMArena Non-English13771219
LMArena Chinese14171215
LMArena French14081261
LMArena German13751222
LMArena Japanese13391132
LMArena Korean13471140
LMArena Russian13811234
LMArena Spanish14201253

Instruction Following Claude Haiku 4.5 leads

Claude Haiku 4.5: 71.4 (#149), Llama 3.1-70B: 65.3 (#223)

BenchmarkClaude Haiku 4.5Llama 3.1-70B
IFEval80.1%82.1%
LMArena Instruction Following14141231

Long Context Claude Haiku 4.5 leads

Claude Haiku 4.5: 43.6 (#92), Llama 3.1-70B: 37.6 (#214)

BenchmarkClaude Haiku 4.5Llama 3.1-70B
LMArena Longer Query14271241

Writing & Preference Claude Haiku 4.5 leads

Claude Haiku 4.5: 57.9 (#123), Llama 3.1-70B: 35.4 (#267)

BenchmarkClaude Haiku 4.5Llama 3.1-70B
LMArena Text13961261
LMArena Creative Writing13721232
WildBench83.9%75.8%
LMArena Multi-Turn14091256
EQ-Bench Creative Writing—784
EQ-Bench 41064—

Frequently asked questions

Is Claude Haiku 4.5 better than Llama 3.1-70B?

Claude Haiku 4.5 is the stronger model overall, scoring 39.5 to 29.6 on the Noometry Index. Llama 3.1-70B costs 5.0× less per token, which makes it the better buy when Claude Haiku 4.5's lead doesn't matter for your workload.

Which is cheaper, Claude Haiku 4.5 or Llama 3.1-70B?

Llama 3.1-70B is cheaper. It lists at $0.40 per million input tokens and $0.40 per million output tokens; Claude Haiku 4.5 lists at $1 and $5.

Is Claude Haiku 4.5 or Llama 3.1-70B better for coding?

Claude Haiku 4.5 scores higher on coding benchmarks: 44.0 versus 30.3 in the Noometry coding category.

Which has the bigger context window?

Claude Haiku 4.5 does, with 200K tokens against 128K.

How many benchmarks do Claude Haiku 4.5 and Llama 3.1-70B share?

30 benchmarks have published results for both models. Claude Haiku 4.5 has 53 scored results on Noometry and Llama 3.1-70B has 35.

Related comparisons

Go deeper