Noometry

Model comparison

Claude 3.7 Sonnet vs Qwen2.5-Max

Qwen2.5-Max is the stronger model overall, scoring 40.7 to 39.5 on the Noometry Index.

Last verified . 27 shared benchmarks.

Claude 3.7 Sonnet Anthropic

39.5

Rank #164 Confirmed

Qwen2.5-Max Alibaba (Qwen)

40.7

Rank #146 Confirmed

Summary

  • They share 27 benchmarks with published results for both. Claude 3.7 Sonnet scores higher in 4 categories and Qwen2.5-Max in 4 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in long context, where Claude 3.7 Sonnet leads 50.3 to 41.4.
  • The biggest single-benchmark swing is LiveBench Reasoning: 87.8% for Claude 3.7 Sonnet and 51.4% for Qwen2.5-Max.

Side by side

Claude 3.7 SonnetQwen2.5-Max
ProviderAnthropicAlibaba (Qwen)
Noometry Index39.540.7
Released2025-02-242025-01-25
WeightsProprietaryProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked5827

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5-Max leads

Claude 3.7 Sonnet: 40.6 (#136), Qwen2.5-Max: 41.8 (#117)

BenchmarkClaude 3.7 SonnetQwen2.5-Max
LiveBench Coding74.5%64.4%
LMArena Coding13611359
SWE-bench Verified61%—
SWE-bench Verified (bash only)52.8%—
Aider Polyglot64.9%—
GSO3.8%—
CadEval54%—

Agentic & Tool Use Not comparable

Claude 3.7 Sonnet: 34.1 (#50), Qwen2.5-Max: —

BenchmarkClaude 3.7 SonnetQwen2.5-Max
TheAgentCompany30.9%—
Cybench20%—
DeepResearch Bench43.6%—
OSWorld35.8%—
METR Time Horizons60%—

Reasoning Qwen2.5-Max leads

Claude 3.7 Sonnet: 18.6 (#277), Qwen2.5-Max: 25.6 (#147)

BenchmarkClaude 3.7 SonnetQwen2.5-Max
LiveBench Reasoning87.8%51.4%
LMArena Hard Prompts13331360
LiveBench Data Analysis74%67.9%
Epoch Capabilities Index141.16132.53
LiveBench76.1%62.3%
ARC-AGI-20.9%—
SimpleBench46.4%—
ARC-AGI-128.6%—
EnigmaEval4.2%—
ForecastBench61.8—

Math Too close to call

Claude 3.7 Sonnet: 37.5 (#153), Qwen2.5-Max: 36.9 (#162)

BenchmarkClaude 3.7 SonnetQwen2.5-Max
LiveBench Math79%58.4%
LMArena Math13371369
OTIS Mock AIME 2024-202557.8%—
Omni-MATH33%—
MATH Level 591.2%—
FrontierMath (Feb 2025 set)4.1%—

Knowledge Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 39.8 (#130), Qwen2.5-Max: 35.3 (#186)

BenchmarkClaude 3.7 SonnetQwen2.5-Max
Confabulations14.7%21.8%
LMArena Expert13211337
GPQA Diamond79.7%—
Humanity's Last Exam8%—
MMLU-Pro78.4%—
GPQA (HELM)60.8%—

Multimodal Not comparable

Claude 3.7 Sonnet: 33.7 (#95), Qwen2.5-Max: —

BenchmarkClaude 3.7 SonnetQwen2.5-Max
LMArena Vision1169—
GeoBench68%—
VPCT39%—
SpatialViz-Bench33.9%—

Multilingual Qwen2.5-Max leads

Claude 3.7 Sonnet: 44.1 (#179), Qwen2.5-Max: 48.1 (#146)

BenchmarkClaude 3.7 SonnetQwen2.5-Max
LMArena Non-English12961352
LMArena Chinese12991382
LMArena French13031396
LMArena German13011350
LMArena Japanese12671300
LMArena Korean12491304
LMArena Russian13111353
LMArena Spanish12981377

Instruction Following Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 72.9 (#125), Qwen2.5-Max: 71.3 (#152)

BenchmarkClaude 3.7 SonnetQwen2.5-Max
LiveBench Instruction Following81.3%75.3%
LMArena Instruction Following13521335
IFEval83.4%—

Long Context Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 50.3 (#10), Qwen2.5-Max: 41.4 (#142)

BenchmarkClaude 3.7 SonnetQwen2.5-Max
LMArena Longer Query13731358
Fiction.LiveBench83.3%—

Writing & Preference Qwen2.5-Max leads

Claude 3.7 Sonnet: 54.4 (#150), Qwen2.5-Max: 55.4 (#146)

BenchmarkClaude 3.7 SonnetQwen2.5-Max
LMArena Text13141367
LMArena Creative Writing13321339
Short-Story Creative Writing81.1%72.9%
LMArena Multi-Turn13391364
LiveBench Language59.9%56.3%
EQ-Bench Creative Writing1412—
WildBench81.4%—

Frequently asked questions

Is Claude 3.7 Sonnet better than Qwen2.5-Max?

Qwen2.5-Max is the stronger model overall, scoring 40.7 to 39.5 on the Noometry Index.

Is Claude 3.7 Sonnet or Qwen2.5-Max better for coding?

Qwen2.5-Max scores higher on coding benchmarks: 41.8 versus 40.6 in the Noometry coding category.

How many benchmarks do Claude 3.7 Sonnet and Qwen2.5-Max share?

27 benchmarks have published results for both models. Claude 3.7 Sonnet has 58 scored results on Noometry and Qwen2.5-Max has 27.

Related comparisons

Go deeper