Noometry

Model comparison

o1-mini vs Olmo 3.1 32b Think

Olmo 3.1 32b Think is the stronger model overall, scoring 37.3 to 33.7 on the Noometry Index.

Last verified . 15 shared benchmarks.

o1-mini OpenAI

33.7

Rank #230 Confirmed

Olmo 3.1 32b Think Allen Institute for AI (Ai2)

37.3

Rank #191 Confirmed

Summary

  • They share 15 benchmarks with published results for both. o1-mini scores higher in 5 categories and Olmo 3.1 32b Think in 3 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Olmo 3.1 32b Think leads 24.1 to 8.9.
  • Olmo 3.1 32b Think has downloadable open weights; the other is API-only.

Side by side

o1-miniOlmo 3.1 32b Think
ProviderOpenAIAllen Institute for AI (Ai2)
Noometry Index33.737.3
Released2024-09-12—
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3915

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Olmo 3.1 32b Think leads

o1-mini: 35.1 (#218), Olmo 3.1 32b Think: 37.1 (#186)

Benchmarko1-miniOlmo 3.1 32b Think
LMArena Coding13621292
Aider Polyglot32.9%—
WeirdML36.3%—
LiveBench Coding48%—
HumanEval+89%—
MBPP+78.8%—

Agentic & Tool Use Not comparable

o1-mini: 24.3 (#112), Olmo 3.1 32b Think: —

Benchmarko1-miniOlmo 3.1 32b Think
Cybench10%—

Reasoning Olmo 3.1 32b Think leads

o1-mini: 8.9 (#340), Olmo 3.1 32b Think: 24.1 (#151)

Benchmarko1-miniOlmo 3.1 32b Think
LMArena Hard Prompts13331272
ARC-AGI-20.8%—
SimpleBench18.1%—
ARC-AGI-114%—
LiveBench Reasoning72.3%—
LiveBench Data Analysis57.9%—
Epoch Capabilities Index135.82—
LiveBench57.8%—

Math Too close to call

o1-mini: 33.7 (#168), Olmo 3.1 32b Think: 33.6 (#170)

Benchmarko1-miniOlmo 3.1 32b Think
LMArena Math13581305
OTIS Mock AIME 2024-202546.9%—
LiveBench Math62%—
MATH Level 589.2%—
FrontierMath (Feb 2025 set)1.7%—

Knowledge Too close to call

o1-mini: 34.8 (#180), Olmo 3.1 32b Think: 35.0 (#178)

Benchmarko1-miniOlmo 3.1 32b Think
LMArena Expert13161294
GPQA Diamond62.4%—
Confabulations18.6%—

Multilingual o1-mini leads

o1-mini: 43.8 (#177), Olmo 3.1 32b Think: 38.3 (#226)

Benchmarko1-miniOlmo 3.1 32b Think
LMArena Non-English12891210
LMArena Chinese13141241
LMArena French12921262
LMArena German12781260
LMArena Russian12831193
LMArena Spanish13021288
LMArena Japanese1245—
LMArena Korean1223—

Instruction Following o1-mini leads

o1-mini: 67.0 (#200), Olmo 3.1 32b Think: 65.8 (#213)

Benchmarko1-miniOlmo 3.1 32b Think
LMArena Instruction Following13041246
LiveBench Instruction Following65.4%—

Long Context o1-mini leads

o1-mini: 40.1 (#156), Olmo 3.1 32b Think: 38.6 (#190)

Benchmarko1-miniOlmo 3.1 32b Think
LMArena Longer Query13211272

Writing & Preference o1-mini leads

o1-mini: 48.6 (#196), Olmo 3.1 32b Think: 46.4 (#215)

Benchmarko1-miniOlmo 3.1 32b Think
LMArena Text13171271
LMArena Creative Writing12441226
LMArena Multi-Turn13141252
Short-Story Creative Writing64.9%—
LiveBench Language40.9%—

Frequently asked questions

Is o1-mini better than Olmo 3.1 32b Think?

Olmo 3.1 32b Think is the stronger model overall, scoring 37.3 to 33.7 on the Noometry Index.

Is o1-mini or Olmo 3.1 32b Think better for coding?

Olmo 3.1 32b Think scores higher on coding benchmarks: 37.1 versus 35.1 in the Noometry coding category.

How many benchmarks do o1-mini and Olmo 3.1 32b Think share?

15 benchmarks have published results for both models. o1-mini has 39 scored results on Noometry and Olmo 3.1 32b Think has 15.

Related comparisons

Go deeper