Noometry

Model comparison

Claude Sonnet 4 vs Qwen3 8B

Claude Sonnet 4 is the stronger model overall, scoring 40.8 to 33.7 on the Noometry Index. Qwen3 8B costs 19× less per token, which makes it the better buy when Claude Sonnet 4's lead doesn't matter for your workload.

Last verified . 9 shared benchmarks.

Claude Sonnet 4 Anthropic

40.8

Rank #145 Confirmed

Qwen3 8B Alibaba (Qwen)

33.7

Rank #238 Confirmed

Summary

  • They share 9 benchmarks with published results for both. Claude Sonnet 4 scores higher in 5 categories and Qwen3 8B in 1 category; 6 gaps are clear of the uncertainty.
  • The widest gap is in coding, where Claude Sonnet 4 leads 43.5 to 34.0.
  • The biggest single-benchmark swing is GPQA Diamond: 79.2% for Claude Sonnet 4 and 56.8% for Qwen3 8B.
  • Qwen3 8B is cheaper at $0.18 / $0.70 per million input/output tokens, against $3 / $15 for Claude Sonnet 4.
  • Claude Sonnet 4 accepts more context: 200K tokens versus 131K.
  • Qwen3 8B has downloadable open weights; the other is API-only.

Side by side

Claude Sonnet 4Qwen3 8B
ProviderAnthropicAlibaba (Qwen)
Noometry Index40.833.7
Released2025-05-222025-04
WeightsProprietaryOpen
Context window200K131K
Max output64K8K
Input $ / M tokens$3$0.18
Output $ / M tokens$15$0.70
Results tracked5811

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude Sonnet 4 leads

Claude Sonnet 4: 43.5 (#88), Qwen3 8B: 34.0 (#248)

BenchmarkClaude Sonnet 4Qwen3 8B
SciCode40%22.6%
SWE-bench Verified (bash only)64.9%—
Aider Polyglot61.3%—
GSO4.9%—
WeirdML46.1%—
LMArena Coding1414—
ALE-Bench655.35—

Agentic & Tool Use Claude Sonnet 4 leads

Claude Sonnet 4: 38.5 (#31), Qwen3 8B: 30.2 (#78)

BenchmarkClaude Sonnet 4Qwen3 8B
Berkeley Function Calling Leaderboard—42.6%
TheAgentCompany33.1%—
Cybench35%—
DeepResearch Bench46.6%—
OSWorld43.9%—
METR Time Horizons62%—

Reasoning Claude Sonnet 4 leads

Claude Sonnet 4: 22.9 (#187), Qwen3 8B: 16.6 (#303)

BenchmarkClaude Sonnet 4Qwen3 8B
CritPt0.3%0%
DTBench77.1%59.7%
LMCA29%8.8%
Epoch Capabilities Index141.69136.17
ARC-AGI-25.9%—
SimpleBench45.5%—
Kagi LLM Benchmark73%—
ARC-AGI-140%—
Chess Puzzles—5%
EnigmaEval3.1%—
LMArena Hard Prompts1372—
ForecastBench60.2—

Math Claude Sonnet 4 leads

Claude Sonnet 4: 43.3 (#80), Qwen3 8B: 34.9 (#191)

BenchmarkClaude Sonnet 4Qwen3 8B
OTIS Mock AIME 2024-202571.1%56.1%
Omni-MATH60.2%—
LMArena Math1375—
MATH Level 584.4%—
FrontierMath (Feb 2025 set)4.1%—
FrontierMath Tier 4 (v1)0%—

Knowledge Claude Sonnet 4 leads

Claude Sonnet 4: 41.8 (#108), Qwen3 8B: 36.1 (#173)

BenchmarkClaude Sonnet 4Qwen3 8B
GPQA Diamond79.2%56.8%
Vectara Hallucination Rate10.3%4.8%
Humanity's Last Exam7.8%—
MMLU-Pro84.3%—
Confabulations13.2%—
GPQA (HELM)70.6%—
LMArena Expert1372—

Multimodal Not comparable

Claude Sonnet 4: 26.2 (#121), Qwen3 8B: —

BenchmarkClaude Sonnet 4Qwen3 8B
LMArena Vision1191—
GeoBench37%—
VPCT34%—
MindCube44.8%—

Multilingual Not comparable

Claude Sonnet 4: 46.7 (#156), Qwen3 8B: —

BenchmarkClaude Sonnet 4Qwen3 8B
LMArena Non-English1333—
LMArena Chinese1350—
LMArena French1363—
LMArena German1331—
LMArena Japanese1302—
LMArena Korean1291—
LMArena Russian1355—
LMArena Spanish1357—

Instruction Following Not comparable

Claude Sonnet 4: 71.7 (#145), Qwen3 8B: —

BenchmarkClaude Sonnet 4Qwen3 8B
IFEval84%—
LMArena Instruction Following1376—

Long Context Qwen3 8B leads

Claude Sonnet 4: 33.7 (#259), Qwen3 8B: 37.9 (#210)

BenchmarkClaude Sonnet 4Qwen3 8B
Fiction.LiveBench46.9%62.1%
LMArena Longer Query1398—

Writing & Preference Not comparable

Claude Sonnet 4: 57.1 (#132), Qwen3 8B: —

BenchmarkClaude Sonnet 4Qwen3 8B
LMArena Text1351—
LMArena Creative Writing1345—
Short-Story Creative Writing81.4%—
EQ-Bench Creative Writing1483—
WildBench83.8%—
LMArena Multi-Turn1376—

Frequently asked questions

Is Claude Sonnet 4 better than Qwen3 8B?

Claude Sonnet 4 is the stronger model overall, scoring 40.8 to 33.7 on the Noometry Index. Qwen3 8B costs 19× less per token, which makes it the better buy when Claude Sonnet 4's lead doesn't matter for your workload.

Which is cheaper, Claude Sonnet 4 or Qwen3 8B?

Qwen3 8B is cheaper. It lists at $0.18 per million input tokens and $0.70 per million output tokens; Claude Sonnet 4 lists at $3 and $15.

Is Claude Sonnet 4 or Qwen3 8B better for coding?

Claude Sonnet 4 scores higher on coding benchmarks: 43.5 versus 34.0 in the Noometry coding category.

Which has the bigger context window?

Claude Sonnet 4 does, with 200K tokens against 131K.

How many benchmarks do Claude Sonnet 4 and Qwen3 8B share?

9 benchmarks have published results for both models. Claude Sonnet 4 has 58 scored results on Noometry and Qwen3 8B has 11.

Related comparisons

Go deeper