Noometry

Model comparison

o3 vs Qwen Max

o3 is the stronger model overall, scoring 47.5 to 34.7 on the Noometry Index.

Last verified . 23 shared benchmarks.

o3 OpenAI

47.5

Rank #61 Confirmed

Qwen Max Alibaba (Qwen)

34.7

Rank #230 Confirmed

Summary

  • They share 23 benchmarks with published results for both. o3 scores higher in 8 categories and Qwen Max in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where o3 leads 50.2 to 22.3.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 84.4% for o3 and 16.1% for Qwen Max.
  • Qwen Max is cheaper at $1.60 / $6.40 per million input/output tokens, against $2 / $8 for o3.
  • o3 accepts more context: 200K tokens versus 33K.

Side by side

o3Qwen Max
ProviderOpenAIAlibaba (Qwen)
Noometry Index47.534.7
Released2025-04-162024-04-03
WeightsProprietaryProprietary
Context window200K33K
Max output100K8K
Input $ / M tokens$2$1.60
Output $ / M tokens$8$6.40
Results tracked6323

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3 leads

o3: 46.8 (#64), Qwen Max: 30.7 (#292)

Benchmarko3Qwen Max
Aider Polyglot81.3%21.8%
LMArena Coding14081288
SWE-bench Verified62.3%—
SWE-bench Verified (bash only)58.4%—
GSO8.8%—
WeirdML52.4%—
CadEval74%—
ALE-Bench933.55—

Agentic & Tool Use Not comparable

o3: 34.5 (#44), Qwen Max: —

Benchmarko3Qwen Max
Berkeley Function Calling Leaderboard63%—
GDPval30.8%—
DeepResearch Bench45.2%—
OSWorld23%—
LMArena Search1144—
METR Time Horizons65.4%—

Reasoning o3 leads

o3: 32.0 (#78), Qwen Max: 25.1 (#151)

Benchmarko3Qwen Max
LMArena Hard Prompts14021269
ARC-AGI-26.5%—
SimpleBench53.1%—
Kagi LLM Benchmark67.6%—
ARC-AGI-160.8%—
CritPt1.4%—
Chess Puzzles38%—
EnigmaEval13.1%—
Mystery Game Puzzles29%—
DTBench84.8%—
LMCA39.7%—
Epoch Capabilities Index146.86—
ForecastBench62.5—

Math o3 leads

o3: 50.2 (#58), Qwen Max: 22.3 (#276)

Benchmarko3Qwen Max
OTIS Mock AIME 2024-202584.4%16.1%
LMArena Math14261275
MATH Level 597.8%67.2%
FrontierMath (Feb 2025 set)18.7%1%
FrontierMath (Tiers 1-3)33.3%—
Omni-MATH71.4%—
FrontierMath Tier 4 (v1)2.1%—

Knowledge o3 leads

o3: 54.6 (#52), Qwen Max: 30.3 (#228)

Benchmarko3Qwen Max
GPQA Diamond81.8%56.1%
LMArena Expert14021248
Humanity's Last Exam20.3%—
SimpleQA Verified49.4%—
MMLU-Pro85.9%—
Confabulations14.4%—
GPQA (HELM)75.3%—

Multimodal Not comparable

o3: 41.4 (#36), Qwen Max: —

Benchmarko3Qwen Max
LMArena Vision1214—
GeoBench74%—
VPCT52%—

Multilingual o3 leads

o3: 51.7 (#105), Qwen Max: 41.8 (#202)

Benchmarko3Qwen Max
LMArena Non-English14011263
LMArena Chinese14371254
LMArena French14301330
LMArena German14201254
LMArena Japanese14031205
LMArena Korean13701142
LMArena Russian14061274
LMArena Spanish13951290

Instruction Following o3 leads

o3: 72.8 (#127), Qwen Max: 66.5 (#208)

Benchmarko3Qwen Max
LMArena Instruction Following13681262
IFEval86.9%—

Long Context o3 leads

o3: 53.3 (#6), Qwen Max: 39.4 (#180)

Benchmarko3Qwen Max
Fiction.LiveBench88.9%66.7%
LMArena Longer Query13721288
CL-bench17.8%—

Writing & Preference o3 leads

o3: 63.5 (#64), Qwen Max: 47.8 (#205)

Benchmarko3Qwen Max
LMArena Text14101282
LMArena Creative Writing13591248
LMArena Multi-Turn14051277
Short-Story Creative Writing83.9%—
EQ-Bench Creative Writing1676—
WildBench86.1%—

Frequently asked questions

Is o3 better than Qwen Max?

o3 is the stronger model overall, scoring 47.5 to 34.7 on the Noometry Index.

Which is cheaper, o3 or Qwen Max?

Qwen Max is cheaper. It lists at $1.60 per million input tokens and $6.40 per million output tokens; o3 lists at $2 and $8.

Is o3 or Qwen Max better for coding?

o3 scores higher on coding benchmarks: 46.8 versus 30.7 in the Noometry coding category.

Which has the bigger context window?

o3 does, with 200K tokens against 33K.

How many benchmarks do o3 and Qwen Max share?

23 benchmarks have published results for both models. o3 has 63 scored results on Noometry and Qwen Max has 23.

Related comparisons

Go deeper