Noometry

Model comparison

Claude 3 Sonnet vs Qwen2.5-VL 72B Instruct

Claude 3 Sonnet and Qwen2.5-VL 72B Instruct score almost the same on the Noometry Index (28.8 vs 29.4), so choose on price, context window or the category you care about most.

Last verified . 1 shared benchmarks.

Claude 3 Sonnet Anthropic

28.8

Rank #311 Confirmed

Qwen2.5-VL 72B Instruct Alibaba (Qwen)

29.4

Rank #298 Confirmed

Summary

  • They share 1 benchmark with published results for both. Claude 3 Sonnet scores higher in 0 categories and Qwen2.5-VL 72B Instruct in 2 categories; one gap is clear of the uncertainty.
  • The widest gap is in multimodal, where Qwen2.5-VL 72B Instruct leads 32.9 to 24.7.
  • Qwen2.5-VL 72B Instruct has downloadable open weights; the other is API-only.

Side by side

Claude 3 SonnetQwen2.5-VL 72B Instruct
ProviderAnthropicAlibaba (Qwen)
Noometry Index28.829.4
Released2024-02-292024-09
WeightsProprietaryOpen
Context window—131K
Max output—8K
Input $ / M tokens—$2.80
Output $ / M tokens—$8.40
Results tracked306

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Claude 3 Sonnet: 29.2 (#296), Qwen2.5-VL 72B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5-VL 72B Instruct
WeirdML10.2%—
BigCodeBench Instruct42.7%—
LMArena Coding1223—
BigCodeBench Complete53.8%—
HumanEval+64%—
MBPP+69.3%—

Agentic & Tool Use Not comparable

Claude 3 Sonnet: —, Qwen2.5-VL 72B Instruct: 18.4 (#136)

BenchmarkClaude 3 SonnetQwen2.5-VL 72B Instruct
OSWorld—5%

Reasoning Too close to call

Claude 3 Sonnet: 19.7 (#232), Qwen2.5-VL 72B Instruct: 20.0 (#228)

BenchmarkClaude 3 SonnetQwen2.5-VL 72B Instruct
Kagi LLM Benchmark—36%
LMArena Hard Prompts1197—
DTBench53.6%—
Epoch Capabilities Index120.69—
WinoGrande75.1%—

Math Not comparable

Claude 3 Sonnet: 10.0 (#305), Qwen2.5-VL 72B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5-VL 72B Instruct
OTIS Mock AIME 2024-20252.5%—
LMArena Math1213—
MATH Level 518.2%—

Knowledge Not comparable

Claude 3 Sonnet: 21.1 (#271), Qwen2.5-VL 72B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5-VL 72B Instruct
GPQA Diamond40.6%—
LMArena Expert1173—
MMLU75.9%—

Multimodal Qwen2.5-VL 72B Instruct leads

Claude 3 Sonnet: 24.7 (#120), Qwen2.5-VL 72B Instruct: 32.9 (#92)

BenchmarkClaude 3 SonnetQwen2.5-VL 72B Instruct
LMArena Vision9841107
Video-MME—73.5%
GeoBench—62%
SpatialViz-Bench—33.3%

Multilingual Not comparable

Claude 3 Sonnet: 38.0 (#229), Qwen2.5-VL 72B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5-VL 72B Instruct
LMArena Non-English1205—
LMArena Chinese1188—
LMArena French1228—
LMArena German1204—
LMArena Japanese1131—
LMArena Korean1128—
LMArena Russian1227—
LMArena Spanish1204—

Instruction Following Not comparable

Claude 3 Sonnet: 63.1 (#230), Qwen2.5-VL 72B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5-VL 72B Instruct
LMArena Instruction Following1200—

Long Context Not comparable

Claude 3 Sonnet: 36.7 (#223), Qwen2.5-VL 72B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5-VL 72B Instruct
LMArena Longer Query1211—

Writing & Preference Not comparable

Claude 3 Sonnet: 42.3 (#233), Qwen2.5-VL 72B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5-VL 72B Instruct
LMArena Text1218—
LMArena Creative Writing1186—
LMArena Multi-Turn1227—

Frequently asked questions

Is Claude 3 Sonnet better than Qwen2.5-VL 72B Instruct?

Claude 3 Sonnet and Qwen2.5-VL 72B Instruct score almost the same on the Noometry Index (28.8 vs 29.4), so choose on price, context window or the category you care about most.

How many benchmarks do Claude 3 Sonnet and Qwen2.5-VL 72B Instruct share?

1 benchmark has published results for both models. Claude 3 Sonnet has 30 scored results on Noometry and Qwen2.5-VL 72B Instruct has 6.

Related comparisons

Go deeper