Noometry

Model comparison

Claude 3.5 Haiku vs Wizardlm 13b

Wizardlm 13b is the stronger model overall, scoring 30.8 to 28.9 on the Noometry Index.

Last verified . 10 shared benchmarks.

Claude 3.5 Haiku Anthropic

28.9

Rank #309 Confirmed

Wizardlm 13b Microsoft

30.8

Rank #272 Confirmed

Summary

  • They share 10 benchmarks with published results for both. Claude 3.5 Haiku scores higher in 5 categories and Wizardlm 13b in 2 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Wizardlm 13b leads 27.8 to 13.7.
  • Wizardlm 13b has downloadable open weights; the other is API-only.

Side by side

Claude 3.5 HaikuWizardlm 13b
ProviderAnthropicMicrosoft
Noometry Index28.930.8
Released2024-10-22—
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked4910

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude 3.5 Haiku leads

Claude 3.5 Haiku: 32.6 (#258), Wizardlm 13b: 29.6 (#293)

BenchmarkClaude 3.5 HaikuWizardlm 13b
LMArena Coding12871035
Aider Polyglot28%—
SciCode27.4%—
WeirdML30.7%—
BigCodeBench Instruct46.1%—
LiveBench Coding51.4%—
BigCodeBench Complete59%—
CadEval32%—

Agentic & Tool Use Not comparable

Claude 3.5 Haiku: 27.7 (#90), Wizardlm 13b: —

BenchmarkClaude 3.5 HaikuWizardlm 13b
BALROG19.3%—

Reasoning Wizardlm 13b leads

Claude 3.5 Haiku: 17.1 (#286), Wizardlm 13b: 18.5 (#265)

BenchmarkClaude 3.5 HaikuWizardlm 13b
LMArena Hard Prompts12511018
CritPt0%—
LiveBench Reasoning28.1%—
DTBench56.7%—
LiveBench Data Analysis48.5%—
Epoch Capabilities Index127.15—
LiveBench43.5%—

Math Wizardlm 13b leads

Claude 3.5 Haiku: 13.7 (#294), Wizardlm 13b: 27.8 (#235)

BenchmarkClaude 3.5 HaikuWizardlm 13b
LMArena Math12441017
OTIS Mock AIME 2024-20254.3%—
Omni-MATH22.4%—
LiveBench Math35.5%—
MATH Level 546.4%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge Not comparable

Claude 3.5 Haiku: 18.7 (#276), Wizardlm 13b: —

BenchmarkClaude 3.5 HaikuWizardlm 13b
GPQA Diamond38.1%—
MMLU-Pro60.5%—
Confabulations36.7%—
GPQA (HELM)36.3%—
LMArena Expert1208—
MMLU74.3%—

Multimodal Not comparable

Claude 3.5 Haiku: 26.3 (#113), Wizardlm 13b: —

BenchmarkClaude 3.5 HaikuWizardlm 13b
LMArena Vision1092—
GeoBench34%—

Multilingual Claude 3.5 Haiku leads

Claude 3.5 Haiku: 40.2 (#213), Wizardlm 13b: 27.3 (#272)

BenchmarkClaude 3.5 HaikuWizardlm 13b
LMArena Non-English12381034
LMArena Chinese12291023
LMArena French1262—
LMArena German1237—
LMArena Japanese1175—
LMArena Korean1172—
LMArena Russian1253—
LMArena Spanish1261—

Instruction Following Claude 3.5 Haiku leads

Claude 3.5 Haiku: 63.1 (#229), Wizardlm 13b: 53.7 (#280)

BenchmarkClaude 3.5 HaikuWizardlm 13b
LMArena Instruction Following12411049
LiveBench Instruction Following61.9%—
IFEval79.2%—

Long Context Claude 3.5 Haiku leads

Claude 3.5 Haiku: 38.3 (#195), Wizardlm 13b: 32.0 (#268)

BenchmarkClaude 3.5 HaikuWizardlm 13b
LMArena Longer Query12611054

Writing & Preference Claude 3.5 Haiku leads

Claude 3.5 Haiku: 42.9 (#228), Wizardlm 13b: 30.7 (#282)

BenchmarkClaude 3.5 HaikuWizardlm 13b
LMArena Text12551077
LMArena Creative Writing12331090
LMArena Multi-Turn12651047
Short-Story Creative Writing73.5%—
EQ-Bench Creative Writing1146—
WildBench76%—
LiveBench Language35.4%—

Frequently asked questions

Is Claude 3.5 Haiku better than Wizardlm 13b?

Wizardlm 13b is the stronger model overall, scoring 30.8 to 28.9 on the Noometry Index.

Is Claude 3.5 Haiku or Wizardlm 13b better for coding?

Claude 3.5 Haiku scores higher on coding benchmarks: 32.6 versus 29.6 in the Noometry coding category.

How many benchmarks do Claude 3.5 Haiku and Wizardlm 13b share?

10 benchmarks have published results for both models. Claude 3.5 Haiku has 49 scored results on Noometry and Wizardlm 13b has 10.

Related comparisons

Go deeper