Noometry

Model comparison

Claude 3.5 Haiku vs QwQ-32B

QwQ-32B is the stronger model overall, scoring 39.8 to 29.2 on the Noometry Index.

Last verified . 33 shared benchmarks.

Claude 3.5 Haiku Anthropic

29.2

Rank #315 Confirmed

QwQ-32B Alibaba (Qwen)

39.8

Rank #159 Confirmed

Summary

  • They share 33 benchmarks with published results for both. Claude 3.5 Haiku scores higher in 0 categories and QwQ-32B in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where QwQ-32B leads 38.0 to 14.7.
  • The biggest single-benchmark swing is LiveBench Reasoning: 28.1% for Claude 3.5 Haiku and 83.5% for QwQ-32B.
  • QwQ-32B has downloadable open weights; the other is API-only.

Side by side

Claude 3.5 HaikuQwQ-32B
ProviderAnthropicAlibaba (Qwen)
Noometry Index29.239.8
Released2024-10-222024-11-28
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked4936

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding QwQ-32B leads

Claude 3.5 Haiku: 32.9 (#265), QwQ-32B: 35.4 (#226)

BenchmarkClaude 3.5 HaikuQwQ-32B
Aider Polyglot28%20.9%
BigCodeBench Instruct46.1%44.6%
LiveBench Coding51.4%72.2%
LMArena Coding12861333
BigCodeBench Complete59%54.4%
SciCode27.4%—
WeirdML30.7%—
CadEval32%—

Agentic & Tool Use Not comparable

Claude 3.5 Haiku: 28.0 (#95), QwQ-32B: —

BenchmarkClaude 3.5 HaikuQwQ-32B
BALROG19.3%—

Reasoning QwQ-32B leads

Claude 3.5 Haiku: 17.7 (#290), QwQ-32B: 23.7 (#174)

BenchmarkClaude 3.5 HaikuQwQ-32B
LiveBench Reasoning28.1%83.5%
LMArena Hard Prompts12511325
LiveBench Data Analysis48.5%65%
Epoch Capabilities Index127.15137.6
LiveBench43.5%72%
CritPt0%—
Chess Puzzles—5%
DTBench56.7%—
ForecastBench—58.3

Math QwQ-32B leads

Claude 3.5 Haiku: 14.7 (#300), QwQ-32B: 38.0 (#143)

BenchmarkClaude 3.5 HaikuQwQ-32B
OTIS Mock AIME 2024-20254.3%59.2%
LiveBench Math35.5%77.8%
LMArena Math12441359
Omni-MATH22.4%—
MATH Level 546.4%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge QwQ-32B leads

Claude 3.5 Haiku: 18.7 (#281), QwQ-32B: 37.2 (#158)

BenchmarkClaude 3.5 HaikuQwQ-32B
GPQA Diamond38.1%65.3%
Confabulations36.7%15.6%
LMArena Expert12081324
MMLU-Pro60.5%—
GPQA (HELM)36.3%—
MMLU74.3%—

Multimodal Not comparable

Claude 3.5 Haiku: 26.8 (#117), QwQ-32B: —

BenchmarkClaude 3.5 HaikuQwQ-32B
LMArena Vision1092—
GeoBench34%—

Multilingual QwQ-32B leads

Claude 3.5 Haiku: 40.0 (#218), QwQ-32B: 44.8 (#176)

BenchmarkClaude 3.5 HaikuQwQ-32B
LMArena Non-English12381305
LMArena Chinese12291378
LMArena French12641336
LMArena German12371313
LMArena Japanese11751262
LMArena Korean11731279
LMArena Russian12531297
LMArena Spanish12611354

Instruction Following QwQ-32B leads

Claude 3.5 Haiku: 62.9 (#234), QwQ-32B: 72.6 (#137)

BenchmarkClaude 3.5 HaikuQwQ-32B
LiveBench Instruction Following61.9%81.8%
LMArena Instruction Following12411297
IFEval79.2%—

Long Context QwQ-32B leads

Claude 3.5 Haiku: 38.3 (#200), QwQ-32B: 49.0 (#11)

BenchmarkClaude 3.5 HaikuQwQ-32B
LMArena Longer Query12611308
Fiction.LiveBench—83.3%

Writing & Preference QwQ-32B leads

Claude 3.5 Haiku: 42.7 (#234), QwQ-32B: 50.6 (#180)

BenchmarkClaude 3.5 HaikuQwQ-32B
LMArena Text12551329
LMArena Creative Writing12331288
Short-Story Creative Writing73.5%80.2%
EQ-Bench Creative Writing11461257
LMArena Multi-Turn12651314
LiveBench Language35.4%51.4%
WildBench76%—

Frequently asked questions

Is Claude 3.5 Haiku better than QwQ-32B?

QwQ-32B is the stronger model overall, scoring 39.8 to 29.2 on the Noometry Index.

Is Claude 3.5 Haiku or QwQ-32B better for coding?

QwQ-32B scores higher on coding benchmarks: 35.4 versus 32.9 in the Noometry coding category.

How many benchmarks do Claude 3.5 Haiku and QwQ-32B share?

33 benchmarks have published results for both models. Claude 3.5 Haiku has 49 scored results on Noometry and QwQ-32B has 36.

Related comparisons

Go deeper