Noometry

Model comparison

Olmo 3 32b Think vs Qwen2.5 72B Instruct

Olmo 3 32b Think is the stronger model overall, scoring 38.1 to 31.5 on the Noometry Index.

Last verified . 14 shared benchmarks.

Olmo 3 32b Think Allen Institute for AI (Ai2)

38.1

Rank #182 Confirmed

Qwen2.5 72B Instruct Alibaba (Qwen)

31.5

Rank #259 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Olmo 3 32b Think scores higher in 8 categories and Qwen2.5 72B Instruct in 0 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in math, where Olmo 3 32b Think leads 33.8 to 18.1.

Side by side

Olmo 3 32b ThinkQwen2.5 72B Instruct
ProviderAllen Institute for AI (Ai2)Alibaba (Qwen)
Noometry Index38.131.5
Released—2024-09
WeightsOpenOpen
Context window—131K
Max output—8K
Input $ / M tokens—$1.40
Output $ / M tokens—$5.60
Results tracked1443

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Olmo 3 32b Think leads

Olmo 3 32b Think: 38.0 (#169), Qwen2.5 72B Instruct: 32.8 (#256)

BenchmarkOlmo 3 32b ThinkQwen2.5 72B Instruct
LMArena Coding13181293
WeirdML—16%
BigCodeBench Instruct—45.8%
BigCodeBench Complete—55.9%

Agentic & Tool Use Not comparable

Olmo 3 32b Think: —, Qwen2.5 72B Instruct: 22.0 (#124)

BenchmarkOlmo 3 32b ThinkQwen2.5 72B Instruct
TheAgentCompany—5.7%
BALROG—16.2%
METR Time Horizons—35.8%

Reasoning Olmo 3 32b Think leads

Olmo 3 32b Think: 24.9 (#138), Qwen2.5 72B Instruct: 21.4 (#197)

BenchmarkOlmo 3 32b ThinkQwen2.5 72B Instruct
LMArena Hard Prompts13021271
DTBench—62.9%
LMCA—13.4%
BIG-Bench Hard—79.8%
Epoch Capabilities Index—129.01
ForecastBench—57.5
HellaSwag—84.8%
PIQA—82.6%
WinoGrande—82.3%

Math Olmo 3 32b Think leads

Olmo 3 32b Think: 33.8 (#165), Qwen2.5 72B Instruct: 18.1 (#281)

BenchmarkOlmo 3 32b ThinkQwen2.5 72B Instruct
LMArena Math13151283
OTIS Mock AIME 2024-2025—8.1%
Omni-MATH—33%
MATH Level 5—63.2%

Knowledge Olmo 3 32b Think leads

Olmo 3 32b Think: 34.4 (#189), Qwen2.5 72B Instruct: 26.9 (#247)

BenchmarkOlmo 3 32b ThinkQwen2.5 72B Instruct
LMArena Expert12721245
GPQA Diamond—49.1%
MMLU-Pro—63.1%
Confabulations—19.1%
GPQA (HELM)—42.6%
ARC (AI2) Challenge—94.5%
MMLU—85.3%
TriviaQA—71.9%

Multilingual Too close to call

Olmo 3 32b Think: 41.4 (#205), Qwen2.5 72B Instruct: 41.2 (#208)

BenchmarkOlmo 3 32b ThinkQwen2.5 72B Instruct
LMArena Non-English12551252
LMArena Chinese13001272
LMArena French12911279
LMArena German12901234
LMArena Russian12551264
LMArena Japanese—1180
LMArena Korean—1188
LMArena Spanish—1255

Instruction Following Olmo 3 32b Think leads

Olmo 3 32b Think: 67.4 (#193), Qwen2.5 72B Instruct: 65.7 (#216)

BenchmarkOlmo 3 32b ThinkQwen2.5 72B Instruct
LMArena Instruction Following12751254
IFEval—80.6%

Long Context Too close to call

Olmo 3 32b Think: 39.4 (#177), Qwen2.5 72B Instruct: 38.9 (#183)

BenchmarkOlmo 3 32b ThinkQwen2.5 72B Instruct
LMArena Longer Query12961282

Writing & Preference Olmo 3 32b Think leads

Olmo 3 32b Think: 49.2 (#188), Qwen2.5 72B Instruct: 46.9 (#210)

BenchmarkOlmo 3 32b ThinkQwen2.5 72B Instruct
LMArena Text13001269
LMArena Creative Writing12561221
LMArena Multi-Turn12891272
WildBench—80.2%

Frequently asked questions

Is Olmo 3 32b Think better than Qwen2.5 72B Instruct?

Olmo 3 32b Think is the stronger model overall, scoring 38.1 to 31.5 on the Noometry Index.

Is Olmo 3 32b Think or Qwen2.5 72B Instruct better for coding?

Olmo 3 32b Think scores higher on coding benchmarks: 38.0 versus 32.8 in the Noometry coding category.

How many benchmarks do Olmo 3 32b Think and Qwen2.5 72B Instruct share?

14 benchmarks have published results for both models. Olmo 3 32b Think has 14 scored results on Noometry and Qwen2.5 72B Instruct has 43.

Related comparisons

Go deeper