Noometry

Model comparison

o1-mini vs Olmo 3.1 32b Instruct

Olmo 3.1 32b Instruct is the stronger model overall, scoring 39.4 to 34.0 on the Noometry Index.

Last verified . 16 shared benchmarks.

o1-mini OpenAI

34.0

Rank #235 Confirmed

Olmo 3.1 32b Instruct Allen Institute for AI (Ai2)

39.4

Rank #168 Confirmed

Summary

  • They share 16 benchmarks with published results for both. o1-mini scores higher in 2 categories and Olmo 3.1 32b Instruct in 6 categories; 5 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Olmo 3.1 32b Instruct leads 26.4 to 8.8.
  • Olmo 3.1 32b Instruct has downloadable open weights; the other is API-only.

Side by side

o1-miniOlmo 3.1 32b Instruct
ProviderOpenAIAllen Institute for AI (Ai2)
Noometry Index34.039.4
Released2024-09-12—
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3916

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Olmo 3.1 32b Instruct leads

o1-mini: 35.5 (#224), Olmo 3.1 32b Instruct: 39.5 (#157)

Benchmarko1-miniOlmo 3.1 32b Instruct
LMArena Coding13621347
Aider Polyglot32.9%—
WeirdML36.3%—
LiveBench Coding48%—
HumanEval+89%—
MBPP+78.8%—

Agentic & Tool Use Not comparable

o1-mini: 24.6 (#118), Olmo 3.1 32b Instruct: —

Benchmarko1-miniOlmo 3.1 32b Instruct
Cybench10%—

Reasoning Olmo 3.1 32b Instruct leads

o1-mini: 8.8 (#346), Olmo 3.1 32b Instruct: 26.4 (#132)

Benchmarko1-miniOlmo 3.1 32b Instruct
LMArena Hard Prompts13331322
ARC-AGI-20.8%—
SimpleBench18.1%—
ARC-AGI-114%—
LiveBench Reasoning72.3%—
LiveBench Data Analysis57.9%—
Epoch Capabilities Index135.82—
LiveBench57.8%—

Math Too close to call

o1-mini: 35.4 (#186), Olmo 3.1 32b Instruct: 36.3 (#167)

Benchmarko1-miniOlmo 3.1 32b Instruct
LMArena Math13581305
OTIS Mock AIME 2024-202546.9%—
LiveBench Math62%—
MATH Level 589.2%—
FrontierMath (Feb 2025 set)1.7%—

Knowledge Olmo 3.1 32b Instruct leads

o1-mini: 34.9 (#192), Olmo 3.1 32b Instruct: 36.1 (#175)

Benchmarko1-miniOlmo 3.1 32b Instruct
LMArena Expert13161308
GPQA Diamond62.4%—
Confabulations18.6%—

Multilingual Too close to call

o1-mini: 43.6 (#182), Olmo 3.1 32b Instruct: 42.6 (#191)

Benchmarko1-miniOlmo 3.1 32b Instruct
LMArena Non-English12891275
LMArena Chinese13141304
LMArena French12931328
LMArena German12781282
LMArena Korean12231206
LMArena Russian12831268
LMArena Spanish13031336
LMArena Japanese1245—

Instruction Following Olmo 3.1 32b Instruct leads

o1-mini: 66.7 (#206), Olmo 3.1 32b Instruct: 68.6 (#187)

Benchmarko1-miniOlmo 3.1 32b Instruct
LMArena Instruction Following13041299
LiveBench Instruction Following65.4%—

Long Context Too close to call

o1-mini: 40.1 (#161), Olmo 3.1 32b Instruct: 39.9 (#166)

Benchmarko1-miniOlmo 3.1 32b Instruct
LMArena Longer Query13201312

Writing & Preference Olmo 3.1 32b Instruct leads

o1-mini: 48.4 (#202), Olmo 3.1 32b Instruct: 50.2 (#185)

Benchmarko1-miniOlmo 3.1 32b Instruct
LMArena Text13171311
LMArena Creative Writing12441264
LMArena Multi-Turn13141309
Short-Story Creative Writing64.9%—
LiveBench Language40.9%—

Frequently asked questions

Is o1-mini better than Olmo 3.1 32b Instruct?

Olmo 3.1 32b Instruct is the stronger model overall, scoring 39.4 to 34.0 on the Noometry Index.

Is o1-mini or Olmo 3.1 32b Instruct better for coding?

Olmo 3.1 32b Instruct scores higher on coding benchmarks: 39.5 versus 35.5 in the Noometry coding category.

How many benchmarks do o1-mini and Olmo 3.1 32b Instruct share?

16 benchmarks have published results for both models. o1-mini has 39 scored results on Noometry and Olmo 3.1 32b Instruct has 16.

Related comparisons

Go deeper