Noometry

Model comparison

Claude 3 Sonnet vs Qwen2.5 32B Instruct

Claude 3 Sonnet and Qwen2.5 32B Instruct score almost the same on the Noometry Index (28.8 vs 29.7), so choose on price, context window or the category you care about most.

Last verified . 6 shared benchmarks.

Claude 3 Sonnet Anthropic

28.8

Rank #311 Confirmed

Qwen2.5 32B Instruct Alibaba (Qwen)

29.7

Rank #291 Confirmed

Summary

  • They share 6 benchmarks with published results for both. Claude 3 Sonnet scores higher in 1 category and Qwen2.5 32B Instruct in 3 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in coding, where Qwen2.5 32B Instruct leads 38.1 to 29.2.
  • The biggest single-benchmark swing is MATH Level 5: 18.2% for Claude 3 Sonnet and 56.1% for Qwen2.5 32B Instruct.
  • Qwen2.5 32B Instruct has downloadable open weights; the other is API-only.

Side by side

Claude 3 SonnetQwen2.5 32B Instruct
ProviderAnthropicAlibaba (Qwen)
Noometry Index28.829.7
Released2024-02-292024-09
WeightsProprietaryOpen
Context window—131K
Max output—8K
Input $ / M tokens—$0.70
Output $ / M tokens—$2.80
Results tracked307

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5 32B Instruct leads

Claude 3 Sonnet: 29.2 (#296), Qwen2.5 32B Instruct: 38.1 (#165)

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
BigCodeBench Instruct42.7%45%
BigCodeBench Complete53.8%52.3%
WeirdML10.2%—
LMArena Coding1223—
HumanEval+64%—
MBPP+69.3%—

Reasoning Claude 3 Sonnet leads

Claude 3 Sonnet: 19.7 (#232), Qwen2.5 32B Instruct: 18.5 (#262)

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
Epoch Capabilities Index120.69128.52
Chess Puzzles—0%
LMArena Hard Prompts1197—
DTBench53.6%—
WinoGrande75.1%—

Math Qwen2.5 32B Instruct leads

Claude 3 Sonnet: 10.0 (#305), Qwen2.5 32B Instruct: 15.3 (#289)

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
OTIS Mock AIME 2024-20252.5%7.4%
MATH Level 518.2%56.1%
LMArena Math1213—

Knowledge Qwen2.5 32B Instruct leads

Claude 3 Sonnet: 21.1 (#271), Qwen2.5 32B Instruct: 24.9 (#259)

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
GPQA Diamond40.6%46.1%
LMArena Expert1173—
MMLU75.9%—

Multimodal Not comparable

Claude 3 Sonnet: 24.7 (#120), Qwen2.5 32B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
LMArena Vision984—

Multilingual Not comparable

Claude 3 Sonnet: 38.0 (#229), Qwen2.5 32B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
LMArena Non-English1205—
LMArena Chinese1188—
LMArena French1228—
LMArena German1204—
LMArena Japanese1131—
LMArena Korean1128—
LMArena Russian1227—
LMArena Spanish1204—

Instruction Following Not comparable

Claude 3 Sonnet: 63.1 (#230), Qwen2.5 32B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
LMArena Instruction Following1200—

Long Context Not comparable

Claude 3 Sonnet: 36.7 (#223), Qwen2.5 32B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
LMArena Longer Query1211—

Writing & Preference Not comparable

Claude 3 Sonnet: 42.3 (#233), Qwen2.5 32B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
LMArena Text1218—
LMArena Creative Writing1186—
LMArena Multi-Turn1227—

Frequently asked questions

Is Claude 3 Sonnet better than Qwen2.5 32B Instruct?

Claude 3 Sonnet and Qwen2.5 32B Instruct score almost the same on the Noometry Index (28.8 vs 29.7), so choose on price, context window or the category you care about most.

Is Claude 3 Sonnet or Qwen2.5 32B Instruct better for coding?

Qwen2.5 32B Instruct scores higher on coding benchmarks: 38.1 versus 29.2 in the Noometry coding category.

How many benchmarks do Claude 3 Sonnet and Qwen2.5 32B Instruct share?

6 benchmarks have published results for both models. Claude 3 Sonnet has 30 scored results on Noometry and Qwen2.5 32B Instruct has 7.

Related comparisons

Go deeper