Noometry

Model comparison

Claude 3.5 Haiku vs Mistral Small 3.2

Mistral Small 3.2 is the stronger model overall, scoring 30.8 to 28.9 on the Noometry Index.

Last verified . 4 shared benchmarks.

Claude 3.5 Haiku Anthropic

28.9

Rank #309 Confirmed

Mistral Small 3.2 Mistral AI

30.8

Rank #271 Confirmed

Summary

  • They share 4 benchmarks with published results for both. Claude 3.5 Haiku scores higher in 0 categories and Mistral Small 3.2 in 4 categories; 3 gaps are clear of the uncertainty.
  • The widest gap is in math, where Mistral Small 3.2 leads 24.9 to 13.7.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 4.3% for Claude 3.5 Haiku and 30.3% for Mistral Small 3.2.
  • Mistral Small 3.2 has downloadable open weights; the other is API-only.

Side by side

Claude 3.5 HaikuMistral Small 3.2
ProviderAnthropicMistral AI
Noometry Index28.930.8
Released2024-10-222025-06-20
WeightsProprietaryOpen
Context window—256K
Max output—16K
Input $ / M tokens—$0.0938
Output $ / M tokens—$0.25
Results tracked496

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Claude 3.5 Haiku: 32.6 (#258), Mistral Small 3.2: —

BenchmarkClaude 3.5 HaikuMistral Small 3.2
Aider Polyglot28%—
SciCode27.4%—
WeirdML30.7%—
BigCodeBench Instruct46.1%—
LiveBench Coding51.4%—
LMArena Coding1287—
BigCodeBench Complete59%—
CadEval32%—

Agentic & Tool Use Not comparable

Claude 3.5 Haiku: 27.7 (#90), Mistral Small 3.2: —

BenchmarkClaude 3.5 HaikuMistral Small 3.2
BALROG19.3%—

Reasoning Too close to call

Claude 3.5 Haiku: 17.1 (#286), Mistral Small 3.2: 17.5 (#282)

BenchmarkClaude 3.5 HaikuMistral Small 3.2
Epoch Capabilities Index127.15131.73
Kagi LLM Benchmark—40.4%
CritPt0%—
Chess Puzzles—1%
LiveBench Reasoning28.1%—
LMArena Hard Prompts1251—
DTBench56.7%—
LiveBench Data Analysis48.5%—
LiveBench43.5%—

Math Mistral Small 3.2 leads

Claude 3.5 Haiku: 13.7 (#294), Mistral Small 3.2: 24.9 (#251)

BenchmarkClaude 3.5 HaikuMistral Small 3.2
OTIS Mock AIME 2024-20254.3%30.3%
Omni-MATH22.4%—
LiveBench Math35.5%—
LMArena Math1244—
MATH Level 546.4%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge Mistral Small 3.2 leads

Claude 3.5 Haiku: 18.7 (#276), Mistral Small 3.2: 26.8 (#249)

BenchmarkClaude 3.5 HaikuMistral Small 3.2
GPQA Diamond38.1%49.1%
MMLU-Pro60.5%—
Confabulations36.7%—
GPQA (HELM)36.3%—
LMArena Expert1208—
MMLU74.3%—

Multimodal Not comparable

Claude 3.5 Haiku: 26.3 (#113), Mistral Small 3.2: —

BenchmarkClaude 3.5 HaikuMistral Small 3.2
LMArena Vision1092—
GeoBench34%—

Multilingual Not comparable

Claude 3.5 Haiku: 40.2 (#213), Mistral Small 3.2: —

BenchmarkClaude 3.5 HaikuMistral Small 3.2
LMArena Non-English1238—
LMArena Chinese1229—
LMArena French1262—
LMArena German1237—
LMArena Japanese1175—
LMArena Korean1172—
LMArena Russian1253—
LMArena Spanish1261—

Instruction Following Not comparable

Claude 3.5 Haiku: 63.1 (#229), Mistral Small 3.2: —

BenchmarkClaude 3.5 HaikuMistral Small 3.2
LiveBench Instruction Following61.9%—
IFEval79.2%—
LMArena Instruction Following1241—

Long Context Not comparable

Claude 3.5 Haiku: 38.3 (#195), Mistral Small 3.2: —

BenchmarkClaude 3.5 HaikuMistral Small 3.2
LMArena Longer Query1261—

Writing & Preference Mistral Small 3.2 leads

Claude 3.5 Haiku: 42.9 (#228), Mistral Small 3.2: 45.1 (#219)

BenchmarkClaude 3.5 HaikuMistral Small 3.2
EQ-Bench Creative Writing11461255
LMArena Text1255—
LMArena Creative Writing1233—
Short-Story Creative Writing73.5%—
WildBench76%—
LMArena Multi-Turn1265—
LiveBench Language35.4%—

Frequently asked questions

Is Claude 3.5 Haiku better than Mistral Small 3.2?

Mistral Small 3.2 is the stronger model overall, scoring 30.8 to 28.9 on the Noometry Index.

How many benchmarks do Claude 3.5 Haiku and Mistral Small 3.2 share?

4 benchmarks have published results for both models. Claude 3.5 Haiku has 49 scored results on Noometry and Mistral Small 3.2 has 6.

Related comparisons

Go deeper