Noometry

Model comparison

Claude 3.5 Haiku vs Mistral Large 4

Mistral Large 4 is the stronger model overall, scoring 43.1 to 29.2 on the Noometry Index.

Last verified . 12 shared benchmarks.

Claude 3.5 Haiku Anthropic

29.2

Rank #315 Confirmed

Mistral Large 4 Mistral AI

43.1

Rank #99 Confirmed

Summary

  • They share 12 benchmarks with published results for both. Claude 3.5 Haiku scores higher in 0 categories and Mistral Large 4 in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Mistral Large 4 leads 40.4 to 14.7.

Side by side

Claude 3.5 HaikuMistral Large 4
ProviderAnthropicMistral AI
Noometry Index29.243.1
Released2024-10-222026-10-06
WeightsProprietaryProprietary
Context window—1.05M
Max output—262K
Input $ / M tokens—$0.68
Output $ / M tokens—$2.09
Results tracked4915

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Large 4 leads

Claude 3.5 Haiku: 32.9 (#265), Mistral Large 4: 48.6 (#57)

BenchmarkClaude 3.5 HaikuMistral Large 4
LMArena Coding12861475
Aider Polyglot28%—
LMArena WebDev—1541
SciCode27.4%—
WeirdML30.7%—
BigCodeBench Instruct46.1%—
LiveBench Coding51.4%—
BigCodeBench Complete59%—
CadEval32%—

Agentic & Tool Use Not comparable

Claude 3.5 Haiku: 28.0 (#95), Mistral Large 4: —

BenchmarkClaude 3.5 HaikuMistral Large 4
BALROG19.3%—

Reasoning Mistral Large 4 leads

Claude 3.5 Haiku: 17.7 (#290), Mistral Large 4: 22.5 (#192)

BenchmarkClaude 3.5 HaikuMistral Large 4
LMArena Hard Prompts12511444
NYT Connections (extended)—27.4%
CritPt0%—
LiveBench Reasoning28.1%—
DTBench56.7%—
LiveBench Data Analysis48.5%—
Epoch Capabilities Index127.15—
LiveBench43.5%—

Math Mistral Large 4 leads

Claude 3.5 Haiku: 14.7 (#300), Mistral Large 4: 40.4 (#91)

BenchmarkClaude 3.5 HaikuMistral Large 4
LMArena Math12441488
OTIS Mock AIME 2024-20254.3%—
Omni-MATH22.4%—
LiveBench Math35.5%—
MATH Level 546.4%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge Mistral Large 4 leads

Claude 3.5 Haiku: 18.7 (#281), Mistral Large 4: 36.6 (#166)

BenchmarkClaude 3.5 HaikuMistral Large 4
LMArena Expert12081447
GPQA Diamond38.1%—
SimpleQA Verified—20%
MMLU-Pro60.5%—
Confabulations36.7%—
GPQA (HELM)36.3%—
MMLU74.3%—

Multimodal Not comparable

Claude 3.5 Haiku: 26.8 (#117), Mistral Large 4: —

BenchmarkClaude 3.5 HaikuMistral Large 4
LMArena Vision1092—
GeoBench34%—

Multilingual Mistral Large 4 leads

Claude 3.5 Haiku: 40.0 (#218), Mistral Large 4: 52.6 (#82)

BenchmarkClaude 3.5 HaikuMistral Large 4
LMArena Non-English12381415
LMArena Chinese12291491
LMArena Russian12531414
LMArena French1264—
LMArena German1237—
LMArena Japanese1175—
LMArena Korean1173—
LMArena Spanish1261—

Instruction Following Mistral Large 4 leads

Claude 3.5 Haiku: 62.9 (#234), Mistral Large 4: 75.0 (#76)

BenchmarkClaude 3.5 HaikuMistral Large 4
LMArena Instruction Following12411424
LiveBench Instruction Following61.9%—
IFEval79.2%—

Long Context Mistral Large 4 leads

Claude 3.5 Haiku: 38.3 (#200), Mistral Large 4: 43.6 (#89)

BenchmarkClaude 3.5 HaikuMistral Large 4
LMArena Longer Query12611429

Writing & Preference Mistral Large 4 leads

Claude 3.5 Haiku: 42.7 (#234), Mistral Large 4: 60.4 (#97)

BenchmarkClaude 3.5 HaikuMistral Large 4
LMArena Text12551427
LMArena Creative Writing12331361
LMArena Multi-Turn12651424
Short-Story Creative Writing73.5%—
EQ-Bench Creative Writing1146—
WildBench76%—
LiveBench Language35.4%—

Frequently asked questions

Is Claude 3.5 Haiku better than Mistral Large 4?

Mistral Large 4 is the stronger model overall, scoring 43.1 to 29.2 on the Noometry Index.

Is Claude 3.5 Haiku or Mistral Large 4 better for coding?

Mistral Large 4 scores higher on coding benchmarks: 48.6 versus 32.9 in the Noometry coding category.

How many benchmarks do Claude 3.5 Haiku and Mistral Large 4 share?

12 benchmarks have published results for both models. Claude 3.5 Haiku has 49 scored results on Noometry and Mistral Large 4 has 15.

Related comparisons

Go deeper