Noometry

Model comparison

GPT-5.1 vs Olmo 3.1 32b Think

GPT-5.1 is the stronger model overall, scoring 49.0 to 37.9 on the Noometry Index.

Last verified . 15 shared benchmarks.

GPT-5.1 OpenAI

49.0

Rank #53 Confirmed

Olmo 3.1 32b Think Allen Institute for AI (Ai2)

37.9

Rank #191 Confirmed

Summary

  • They share 15 benchmarks with published results for both. GPT-5.1 scores higher in 8 categories and Olmo 3.1 32b Think in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where GPT-5.1 leads 64.5 to 46.2.
  • Olmo 3.1 32b Think has downloadable open weights; the other is API-only.

Side by side

GPT-5.1Olmo 3.1 32b Think
ProviderOpenAIAllen Institute for AI (Ai2)
Noometry Index49.037.9
Released2025-11-13—
WeightsProprietaryOpen
Context window400K—
Max output128K—
Input $ / M tokens$1.25—
Output $ / M tokens$10—
Results tracked6315

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.1 leads

GPT-5.1: 46.4 (#66), Olmo 3.1 32b Think: 37.7 (#189)

BenchmarkGPT-5.1Olmo 3.1 32b Think
LMArena Coding14541291
SWE-bench Verified68%—
SWE-bench Verified (bash only)66%—
LMArena WebDev1395—
SciCode43.3%—
GSO13.7%—
WeirdML60.8%—
LiveBench Coding72.5%—
ALE-Bench1,192—

Agentic & Tool Use Not comparable

GPT-5.1: 32.7 (#60), Olmo 3.1 32b Think: —

BenchmarkGPT-5.1Olmo 3.1 32b Think
Terminal-Bench47.6%—
DeepResearch Bench42.8%—
LMArena Search1199—
Vending-Bench 21,473—

Reasoning GPT-5.1 leads

GPT-5.1: 39.8 (#58), Olmo 3.1 32b Think: 25.2 (#150)

BenchmarkGPT-5.1Olmo 3.1 32b Think
LMArena Hard Prompts14571272
ARC-AGI-217.6%—
SimpleBench53.2%—
ARC-AGI-172.8%—
CritPt4.9%—
Chess Puzzles32%—
EnigmaEval11.2%—
LiveBench Reasoning95.8%—
Mystery Game Puzzles19%—
DTBench90.1%—
LiveBench Data Analysis72.1%—
LMCA43.9%—
Epoch Capabilities Index149.64—
ForecastBench58.1—
LiveBench78.8%—

Math GPT-5.1 leads

GPT-5.1: 52.2 (#51), Olmo 3.1 32b Think: 36.3 (#168)

BenchmarkGPT-5.1Olmo 3.1 32b Think
LMArena Math14471305
OTIS Mock AIME 2024-202588.6%—
Omni-MATH46.4%—
LiveBench Math94.5%—
FrontierMath (Feb 2025 set)31%—
FrontierMath Tier 4 (v1)12.5%—

Knowledge GPT-5.1 leads

GPT-5.1: 50.6 (#71), Olmo 3.1 32b Think: 35.7 (#181)

BenchmarkGPT-5.1Olmo 3.1 32b Think
LMArena Expert14701295
GPQA Diamond87.6%—
Humanity's Last Exam23.7%—
SimpleQA Verified48%—
MMLU-Pro57.9%—
Vectara Hallucination Rate10.9%—
GPQA (HELM)44.2%—

Multimodal Not comparable

GPT-5.1: 44.8 (#19), Olmo 3.1 32b Think: —

BenchmarkGPT-5.1Olmo 3.1 32b Think
LMArena Vision1250—
VPCT58.7%—
LMArena Document1403—

Multilingual GPT-5.1 leads

GPT-5.1: 53.8 (#56), Olmo 3.1 32b Think: 38.1 (#231)

BenchmarkGPT-5.1Olmo 3.1 32b Think
LMArena Non-English14311209
LMArena Chinese14951242
LMArena French14501260
LMArena German14381262
LMArena Russian14351193
LMArena Spanish14331289
LMArena Japanese1453—
LMArena Korean1401—

Instruction Following GPT-5.1 leads

GPT-5.1: 83.9 (#1), Olmo 3.1 32b Think: 65.6 (#218)

BenchmarkGPT-5.1Olmo 3.1 32b Think
LMArena Instruction Following14431247
LiveBench Instruction Following93.3%—
IFEval93.5%—

Long Context GPT-5.1 leads

GPT-5.1: 47.6 (#14), Olmo 3.1 32b Think: 38.6 (#195)

BenchmarkGPT-5.1Olmo 3.1 32b Think
LMArena Longer Query14471272
CL-bench23.7%—
CL-bench Life17.3%—

Writing & Preference GPT-5.1 leads

GPT-5.1: 64.5 (#55), Olmo 3.1 32b Think: 46.2 (#220)

BenchmarkGPT-5.1Olmo 3.1 32b Think
LMArena Text14431272
LMArena Creative Writing14271226
LMArena Multi-Turn14501252
WildBench86.3%—
LiveBench Language80.2%—

Frequently asked questions

Is GPT-5.1 better than Olmo 3.1 32b Think?

GPT-5.1 is the stronger model overall, scoring 49.0 to 37.9 on the Noometry Index.

Is GPT-5.1 or Olmo 3.1 32b Think better for coding?

GPT-5.1 scores higher on coding benchmarks: 46.4 versus 37.7 in the Noometry coding category.

How many benchmarks do GPT-5.1 and Olmo 3.1 32b Think share?

15 benchmarks have published results for both models. GPT-5.1 has 63 scored results on Noometry and Olmo 3.1 32b Think has 15.

Related comparisons

Go deeper