Noometry

Model comparison

Claude 3.5 Sonnet vs Qwen3-4B

Claude 3.5 Sonnet is the stronger model overall, scoring 34.4 to 31.5 on the Noometry Index.

Last verified . 2 shared benchmarks.

Claude 3.5 Sonnet Anthropic

34.4

Rank #225 Confirmed

Qwen3-4B Alibaba (Qwen)

31.5

Rank #261 Confirmed

Summary

  • They share 2 benchmarks with published results for both. Claude 3.5 Sonnet scores higher in 2 categories and Qwen3-4B in 2 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in math, where Qwen3-4B leads 28.5 to 18.0.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 8.5% for Claude 3.5 Sonnet and 52.2% for Qwen3-4B.
  • Qwen3-4B has downloadable open weights; the other is API-only.

Side by side

Claude 3.5 SonnetQwen3-4B
ProviderAnthropicAlibaba (Qwen)
Noometry Index34.431.5
Released2024-06-202025-04-29
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked606

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Claude 3.5 Sonnet: 39.5 (#145), Qwen3-4B: —

BenchmarkClaude 3.5 SonnetQwen3-4B
Aider Polyglot51.6%—
GSO4.6%—
WeirdML40%—
BigCodeBench Instruct46.8%—
LiveBench Coding67.1%—
LMArena Coding1343—
BigCodeBench Complete58.6%—
CadEval48%—
HumanEval+81.7%—
MBPP+74.3%—

Agentic & Tool Use Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 32.1 (#61), Qwen3-4B: 27.3 (#94)

BenchmarkClaude 3.5 SonnetQwen3-4B
Berkeley Function Calling Leaderboard—35.7%
TheAgentCompany24%—
Cybench17.5%—
BALROG32.6%—
METR Time Horizons45.2%—

Reasoning Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 22.5 (#176), Qwen3-4B: 18.5 (#264)

BenchmarkClaude 3.5 SonnetQwen3-4B
SimpleBench41.4%—
Chess Puzzles—4%
EnigmaEval0.9%—
LiveBench Reasoning56.7%—
LMArena Hard Prompts1305—
DTBench67.8%—
LiveBench Data Analysis55%—
Epoch Capabilities Index133.55—
ForecastBench60.7—
LiveBench59%—

Math Qwen3-4B leads

Claude 3.5 Sonnet: 18.0 (#282), Qwen3-4B: 28.5 (#228)

BenchmarkClaude 3.5 SonnetQwen3-4B
OTIS Mock AIME 2024-20258.5%52.2%
MathArena Final-Answer Competitions—38.5%
Omni-MATH27.6%—
LiveBench Math52.3%—
LMArena Math1306—
MATH Level 556.9%—
FrontierMath (Feb 2025 set)2.1%—
FrontierMath Tier 4 (v1)0%—

Knowledge Qwen3-4B leads

Claude 3.5 Sonnet: 28.5 (#236), Qwen3-4B: 33.0 (#198)

BenchmarkClaude 3.5 SonnetQwen3-4B
GPQA Diamond55.3%52.3%
Humanity's Last Exam4.1%—
MMLU-Pro77.7%—
Confabulations19.9%—
Vectara Hallucination Rate—5.7%
GPQA (HELM)56.5%—
LMArena Expert1264—
MMLU87.3%—

Multimodal Not comparable

Claude 3.5 Sonnet: 26.0 (#115), Qwen3-4B: —

BenchmarkClaude 3.5 SonnetQwen3-4B
LMArena Vision1125—
Video-MME60%—
GeoBench62%—
VPCT33%—

Multilingual Not comparable

Claude 3.5 Sonnet: 43.4 (#180), Qwen3-4B: —

BenchmarkClaude 3.5 SonnetQwen3-4B
LMArena Non-English1283—
LMArena Chinese1271—
LMArena French1304—
LMArena German1297—
LMArena Japanese1234—
LMArena Korean1200—
LMArena Russian1305—
LMArena Spanish1289—

Instruction Following Not comparable

Claude 3.5 Sonnet: 69.0 (#177), Qwen3-4B: —

BenchmarkClaude 3.5 SonnetQwen3-4B
LiveBench Instruction Following69.3%—
IFEval85.5%—
LMArena Instruction Following1297—

Long Context Not comparable

Claude 3.5 Sonnet: 39.9 (#162), Qwen3-4B: —

BenchmarkClaude 3.5 SonnetQwen3-4B
LMArena Longer Query1311—

Writing & Preference Not comparable

Claude 3.5 Sonnet: 53.0 (#159), Qwen3-4B: —

BenchmarkClaude 3.5 SonnetQwen3-4B
LMArena Text1298—
LMArena Creative Writing1292—
Short-Story Creative Writing80.3%—
EQ-Bench Creative Writing1451—
WildBench79.2%—
LMArena Multi-Turn1326—
LiveBench Language53.8%—

Frequently asked questions

Is Claude 3.5 Sonnet better than Qwen3-4B?

Claude 3.5 Sonnet is the stronger model overall, scoring 34.4 to 31.5 on the Noometry Index.

How many benchmarks do Claude 3.5 Sonnet and Qwen3-4B share?

2 benchmarks have published results for both models. Claude 3.5 Sonnet has 60 scored results on Noometry and Qwen3-4B has 6.

Related comparisons

Go deeper