Noometry

Model comparison

o1 vs Olmo 3.1 32b Think

o1 is the stronger model overall, scoring 40.9 to 37.9 on the Noometry Index.

Last verified . 15 shared benchmarks.

o1 OpenAI

40.9

Rank #143 Confirmed

Olmo 3.1 32b Think Allen Institute for AI (Ai2)

37.9

Rank #191 Confirmed

Summary

  • They share 15 benchmarks with published results for both. o1 scores higher in 7 categories and Olmo 3.1 32b Think in 1 category; 7 gaps are clear of the uncertainty.
  • The widest gap is in long context, where o1 leads 50.3 to 38.6.
  • Olmo 3.1 32b Think has downloadable open weights; the other is API-only.

Side by side

o1Olmo 3.1 32b Think
ProviderOpenAIAllen Institute for AI (Ai2)
Noometry Index40.937.9
Released2024-09-12—
WeightsProprietaryOpen
Context window200K—
Max output100K—
Input $ / M tokens$15—
Output $ / M tokens$60—
Results tracked5215

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o1 leads

o1: 46.1 (#70), Olmo 3.1 32b Think: 37.7 (#189)

Benchmarko1Olmo 3.1 32b Think
LMArena Coding13671291
Aider Polyglot61.7%—
WeirdML47.6%—
LiveBench Coding69.7%—
CadEval56%—
HumanEval+89%—
MBPP+80.2%—

Agentic & Tool Use Not comparable

o1: 24.6 (#117), Olmo 3.1 32b Think: —

Benchmarko1Olmo 3.1 32b Think
Cybench10%—
METR Time Horizons51.1%—

Reasoning o1 leads

o1: 27.9 (#111), Olmo 3.1 32b Think: 25.2 (#150)

Benchmarko1Olmo 3.1 32b Think
LMArena Hard Prompts13711272
SimpleBench41.7%—
ARC-AGI-130.7%—
Chess Puzzles15%—
EnigmaEval5.7%—
LiveBench Reasoning91.6%—
DTBench74.7%—
LiveBench Data Analysis65.5%—
LMCA22.3%—
Epoch Capabilities Index141.91—
LiveBench75.7%—

Math Too close to call

o1: 36.1 (#175), Olmo 3.1 32b Think: 36.3 (#168)

Benchmarko1Olmo 3.1 32b Think
LMArena Math13881305
FrontierMath (Tiers 1-3)14.7%—
OTIS Mock AIME 2024-202573.3%—
LiveBench Math80.3%—
MATH Level 594.7%—
FrontierMath (Feb 2025 set)9.3%—

Knowledge o1 leads

o1: 41.5 (#110), Olmo 3.1 32b Think: 35.7 (#181)

Benchmarko1Olmo 3.1 32b Think
LMArena Expert13611295
GPQA Diamond76.8%—
Humanity's Last Exam8%—
SimpleQA Verified41.1%—
Confabulations11.7%—

Multimodal Not comparable

o1: 34.2 (#93), Olmo 3.1 32b Think: —

Benchmarko1Olmo 3.1 32b Think
LMArena Vision1168—
GeoBench80%—
VPCT37%—
SpatialViz-Bench41.4%—

Multilingual o1 leads

o1: 48.6 (#142), Olmo 3.1 32b Think: 38.1 (#231)

Benchmarko1Olmo 3.1 32b Think
LMArena Non-English13581209
LMArena Chinese13941242
LMArena French13441260
LMArena German13371262
LMArena Russian13561193
LMArena Spanish13451289
LMArena Japanese1346—
LMArena Korean1396—

Instruction Following o1 leads

o1: 74.8 (#86), Olmo 3.1 32b Think: 65.6 (#218)

Benchmarko1Olmo 3.1 32b Think
LMArena Instruction Following13671247
LiveBench Instruction Following81.5%—

Long Context o1 leads

o1: 50.3 (#9), Olmo 3.1 32b Think: 38.6 (#195)

Benchmarko1Olmo 3.1 32b Think
LMArena Longer Query13781272
Fiction.LiveBench83.3%—

Writing & Preference o1 leads

o1: 55.6 (#144), Olmo 3.1 32b Think: 46.2 (#220)

Benchmarko1Olmo 3.1 32b Think
LMArena Text13661272
LMArena Creative Writing13481226
LMArena Multi-Turn13691252
Short-Story Creative Writing70.2%—
LiveBench Language65.4%—

Frequently asked questions

Is o1 better than Olmo 3.1 32b Think?

o1 is the stronger model overall, scoring 40.9 to 37.9 on the Noometry Index.

Is o1 or Olmo 3.1 32b Think better for coding?

o1 scores higher on coding benchmarks: 46.1 versus 37.7 in the Noometry coding category.

How many benchmarks do o1 and Olmo 3.1 32b Think share?

15 benchmarks have published results for both models. o1 has 52 scored results on Noometry and Olmo 3.1 32b Think has 15.

Related comparisons

Go deeper