Noometry

Model comparison

o1 vs Qwen Max

o1 is the stronger model overall, scoring 40.6 to 34.4 on the Noometry Index. Qwen Max costs 9.4× less per token, which makes it the better buy when o1's lead doesn't matter for your workload.

Last verified . 23 shared benchmarks.

o1 OpenAI

40.6

Rank #140 Confirmed

Qwen Max Alibaba (Qwen)

34.4

Rank #226 Confirmed

Summary

  • They share 23 benchmarks with published results for both. o1 scores higher in 8 categories and Qwen Max in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in coding, where o1 leads 45.7 to 30.3.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 73.3% for o1 and 16.1% for Qwen Max.
  • Qwen Max is cheaper at $1.60 / $6.40 per million input/output tokens, against $15 / $60 for o1.
  • o1 accepts more context: 200K tokens versus 33K.

Side by side

o1Qwen Max
ProviderOpenAIAlibaba (Qwen)
Noometry Index40.634.4
Released2024-09-122024-04-03
WeightsProprietaryProprietary
Context window200K33K
Max output100K8K
Input $ / M tokens$15$1.60
Output $ / M tokens$60$6.40
Results tracked5223

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o1 leads

o1: 45.7 (#64), Qwen Max: 30.3 (#288)

Benchmarko1Qwen Max
Aider Polyglot61.7%21.8%
LMArena Coding13671288
WeirdML47.6%—
LiveBench Coding69.7%—
CadEval56%—
HumanEval+89%—
MBPP+80.2%—

Agentic & Tool Use Not comparable

o1: 24.3 (#111), Qwen Max: —

Benchmarko1Qwen Max
Cybench10%—
METR Time Horizons51.1%—

Reasoning o1 leads

o1: 27.5 (#103), Qwen Max: 24.1 (#154)

Benchmarko1Qwen Max
LMArena Hard Prompts13721270
SimpleBench41.7%—
ARC-AGI-130.7%—
Chess Puzzles15%—
EnigmaEval5.7%—
LiveBench Reasoning91.6%—
DTBench74.7%—
LiveBench Data Analysis65.5%—
LMCA22.3%—
Epoch Capabilities Index141.92—
LiveBench75.7%—

Math o1 leads

o1: 34.9 (#150), Qwen Max: 21.0 (#272)

Benchmarko1Qwen Max
OTIS Mock AIME 2024-202573.3%16.1%
LMArena Math13881275
MATH Level 594.7%67.2%
FrontierMath (Feb 2025 set)9.3%1%
FrontierMath (Tiers 1-3)14.7%—
LiveBench Math80.3%—

Knowledge o1 leads

o1: 41.5 (#101), Qwen Max: 30.3 (#222)

Benchmarko1Qwen Max
GPQA Diamond76.8%56.1%
LMArena Expert13621248
Humanity's Last Exam8%—
SimpleQA Verified41.1%—
Confabulations11.7%—

Multimodal Not comparable

o1: 33.6 (#87), Qwen Max: —

Benchmarko1Qwen Max
LMArena Vision1168—
GeoBench80%—
VPCT37%—
SpatialViz-Bench41.4%—

Multilingual o1 leads

o1: 48.8 (#137), Qwen Max: 42.0 (#197)

Benchmarko1Qwen Max
LMArena Non-English13591263
LMArena Chinese13941254
LMArena French13431329
LMArena German13371254
LMArena Japanese13461205
LMArena Korean13961141
LMArena Russian13561274
LMArena Spanish13451290

Instruction Following o1 leads

o1: 75.0 (#82), Qwen Max: 66.7 (#203)

Benchmarko1Qwen Max
LMArena Instruction Following13681262
LiveBench Instruction Following81.5%—

Long Context o1 leads

o1: 50.3 (#9), Qwen Max: 39.4 (#175)

Benchmarko1Qwen Max
Fiction.LiveBench83.3%66.7%
LMArena Longer Query13781289

Writing & Preference o1 leads

o1: 55.8 (#139), Qwen Max: 48.0 (#200)

Benchmarko1Qwen Max
LMArena Text13661282
LMArena Creative Writing13471248
LMArena Multi-Turn13681277
Short-Story Creative Writing70.2%—
LiveBench Language65.4%—

Frequently asked questions

Is o1 better than Qwen Max?

o1 is the stronger model overall, scoring 40.6 to 34.4 on the Noometry Index. Qwen Max costs 9.4× less per token, which makes it the better buy when o1's lead doesn't matter for your workload.

Which is cheaper, o1 or Qwen Max?

Qwen Max is cheaper. It lists at $1.60 per million input tokens and $6.40 per million output tokens; o1 lists at $15 and $60.

Is o1 or Qwen Max better for coding?

o1 scores higher on coding benchmarks: 45.7 versus 30.3 in the Noometry coding category.

Which has the bigger context window?

o1 does, with 200K tokens against 33K.

How many benchmarks do o1 and Qwen Max share?

23 benchmarks have published results for both models. o1 has 52 scored results on Noometry and Qwen Max has 23.

Related comparisons

Go deeper