Noometry

Model comparison

o1-mini vs Qwen2.5 32B Instruct

o1-mini is the stronger model overall, scoring 33.7 to 29.7 on the Noometry Index.

Last verified . 4 shared benchmarks.

o1-mini OpenAI

33.7

Rank #230 Confirmed

Qwen2.5 32B Instruct Alibaba (Qwen)

29.7

Rank #291 Confirmed

Summary

  • They share 4 benchmarks with published results for both. o1-mini scores higher in 2 categories and Qwen2.5 32B Instruct in 2 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in math, where o1-mini leads 33.7 to 15.3.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 46.9% for o1-mini and 7.4% for Qwen2.5 32B Instruct.
  • Qwen2.5 32B Instruct has downloadable open weights; the other is API-only.

Side by side

o1-miniQwen2.5 32B Instruct
ProviderOpenAIAlibaba (Qwen)
Noometry Index33.729.7
Released2024-09-122024-09
WeightsProprietaryOpen
Context window—131K
Max output—8K
Input $ / M tokens—$0.70
Output $ / M tokens—$2.80
Results tracked397

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5 32B Instruct leads

o1-mini: 35.1 (#218), Qwen2.5 32B Instruct: 38.1 (#165)

Benchmarko1-miniQwen2.5 32B Instruct
Aider Polyglot32.9%—
WeirdML36.3%—
BigCodeBench Instruct—45%
LiveBench Coding48%—
LMArena Coding1362—
BigCodeBench Complete—52.3%
HumanEval+89%—
MBPP+78.8%—

Agentic & Tool Use Not comparable

o1-mini: 24.3 (#112), Qwen2.5 32B Instruct: —

Benchmarko1-miniQwen2.5 32B Instruct
Cybench10%—

Reasoning Qwen2.5 32B Instruct leads

o1-mini: 8.9 (#340), Qwen2.5 32B Instruct: 18.5 (#262)

Benchmarko1-miniQwen2.5 32B Instruct
Epoch Capabilities Index135.82128.52
ARC-AGI-20.8%—
SimpleBench18.1%—
ARC-AGI-114%—
Chess Puzzles—0%
LiveBench Reasoning72.3%—
LMArena Hard Prompts1333—
LiveBench Data Analysis57.9%—
LiveBench57.8%—

Math o1-mini leads

o1-mini: 33.7 (#168), Qwen2.5 32B Instruct: 15.3 (#289)

Benchmarko1-miniQwen2.5 32B Instruct
OTIS Mock AIME 2024-202546.9%7.4%
MATH Level 589.2%56.1%
LiveBench Math62%—
LMArena Math1358—
FrontierMath (Feb 2025 set)1.7%—

Knowledge o1-mini leads

o1-mini: 34.8 (#180), Qwen2.5 32B Instruct: 24.9 (#259)

Benchmarko1-miniQwen2.5 32B Instruct
GPQA Diamond62.4%46.1%
Confabulations18.6%—
LMArena Expert1316—

Multilingual Not comparable

o1-mini: 43.8 (#177), Qwen2.5 32B Instruct: —

Benchmarko1-miniQwen2.5 32B Instruct
LMArena Non-English1289—
LMArena Chinese1314—
LMArena French1292—
LMArena German1278—
LMArena Japanese1245—
LMArena Korean1223—
LMArena Russian1283—
LMArena Spanish1302—

Instruction Following Not comparable

o1-mini: 67.0 (#200), Qwen2.5 32B Instruct: —

Benchmarko1-miniQwen2.5 32B Instruct
LiveBench Instruction Following65.4%—
LMArena Instruction Following1304—

Long Context Not comparable

o1-mini: 40.1 (#156), Qwen2.5 32B Instruct: —

Benchmarko1-miniQwen2.5 32B Instruct
LMArena Longer Query1321—

Writing & Preference Not comparable

o1-mini: 48.6 (#196), Qwen2.5 32B Instruct: —

Benchmarko1-miniQwen2.5 32B Instruct
LMArena Text1317—
LMArena Creative Writing1244—
Short-Story Creative Writing64.9%—
LMArena Multi-Turn1314—
LiveBench Language40.9%—

Frequently asked questions

Is o1-mini better than Qwen2.5 32B Instruct?

o1-mini is the stronger model overall, scoring 33.7 to 29.7 on the Noometry Index.

Is o1-mini or Qwen2.5 32B Instruct better for coding?

Qwen2.5 32B Instruct scores higher on coding benchmarks: 38.1 versus 35.1 in the Noometry coding category.

How many benchmarks do o1-mini and Qwen2.5 32B Instruct share?

4 benchmarks have published results for both models. o1-mini has 39 scored results on Noometry and Qwen2.5 32B Instruct has 7.

Related comparisons

Go deeper