Noometry

Model comparison

Llama 4 Scout vs Olmo 3.1 32b Think

Olmo 3.1 32b Think is the stronger model overall, scoring 37.9 to 27.7 on the Noometry Index.

Last verified . 15 shared benchmarks.

Llama 4 Scout Meta

27.7

Rank #330 Confirmed

Olmo 3.1 32b Think Allen Institute for AI (Ai2)

37.9

Rank #191 Confirmed

Summary

  • They share 15 benchmarks with published results for both. Llama 4 Scout scores higher in 2 categories and Olmo 3.1 32b Think in 6 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in coding, where Olmo 3.1 32b Think leads 37.7 to 20.2.

Side by side

Llama 4 ScoutOlmo 3.1 32b Think
ProviderMetaAllen Institute for AI (Ai2)
Noometry Index27.737.9
Released2025-04-05—
WeightsOpenOpen
Context window128K—
Max output4K—
Input $ / M tokens$0.10—
Output $ / M tokens$0.30—
Results tracked4315

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Olmo 3.1 32b Think leads

Llama 4 Scout: 20.2 (#339), Olmo 3.1 32b Think: 37.7 (#189)

BenchmarkLlama 4 ScoutOlmo 3.1 32b Think
LMArena Coding12861291
SWE-bench Verified (bash only)9.1%—
SciCode17%—
BigCodeBench Complete43.1%—

Agentic & Tool Use Not comparable

Llama 4 Scout: 24.6 (#119), Olmo 3.1 32b Think: —

BenchmarkLlama 4 ScoutOlmo 3.1 32b Think
Berkeley Function Calling Leaderboard28.1%—

Reasoning Olmo 3.1 32b Think leads

Llama 4 Scout: 9.1 (#345), Olmo 3.1 32b Think: 25.2 (#150)

BenchmarkLlama 4 ScoutOlmo 3.1 32b Think
LMArena Hard Prompts12661272
ARC-AGI-20%—
Kagi LLM Benchmark36.9%—
ARC-AGI-10.5%—
CritPt0%—
DTBench57.9%—
LMCA12%—
Epoch Capabilities Index129.64—
ForecastBench57.5—

Math Olmo 3.1 32b Think leads

Llama 4 Scout: 19.6 (#286), Olmo 3.1 32b Think: 36.3 (#168)

BenchmarkLlama 4 ScoutOlmo 3.1 32b Think
LMArena Math12871305
OTIS Mock AIME 2024-20257.8%—
Omni-MATH37.3%—
MATH Level 562.3%—
FrontierMath (Feb 2025 set)0%—

Knowledge Olmo 3.1 32b Think leads

Llama 4 Scout: 31.9 (#217), Olmo 3.1 32b Think: 35.7 (#181)

BenchmarkLlama 4 ScoutOlmo 3.1 32b Think
LMArena Expert12351295
GPQA Diamond51.8%—
MMLU-Pro74.2%—
Vectara Hallucination Rate7.7%—
GPQA (HELM)50.7%—

Multimodal Not comparable

Llama 4 Scout: 32.2 (#102), Olmo 3.1 32b Think: —

BenchmarkLlama 4 ScoutOlmo 3.1 32b Think
LMArena Vision1118—
SpatialViz-Bench34.2%—

Multilingual Llama 4 Scout leads

Llama 4 Scout: 41.0 (#212), Olmo 3.1 32b Think: 38.1 (#231)

BenchmarkLlama 4 ScoutOlmo 3.1 32b Think
LMArena Non-English12521209
LMArena Chinese12551242
LMArena French12821260
LMArena German12721262
LMArena Russian12631193
LMArena Spanish12781289
LMArena Japanese1206—
LMArena Korean1207—

Instruction Following Too close to call

Llama 4 Scout: 65.8 (#217), Olmo 3.1 32b Think: 65.6 (#218)

BenchmarkLlama 4 ScoutOlmo 3.1 32b Think
LMArena Instruction Following12481247
IFEval81.8%—

Long Context Olmo 3.1 32b Think leads

Llama 4 Scout: 27.5 (#294), Olmo 3.1 32b Think: 38.6 (#195)

BenchmarkLlama 4 ScoutOlmo 3.1 32b Think
LMArena Longer Query12651272
Fiction.LiveBench36%—

Writing & Preference Olmo 3.1 32b Think leads

Llama 4 Scout: 37.0 (#261), Olmo 3.1 32b Think: 46.2 (#220)

BenchmarkLlama 4 ScoutOlmo 3.1 32b Think
LMArena Text12791272
LMArena Creative Writing12491226
LMArena Multi-Turn12801252
EQ-Bench Creative Writing783—
WildBench78%—

Frequently asked questions

Is Llama 4 Scout better than Olmo 3.1 32b Think?

Olmo 3.1 32b Think is the stronger model overall, scoring 37.9 to 27.7 on the Noometry Index.

Is Llama 4 Scout or Olmo 3.1 32b Think better for coding?

Olmo 3.1 32b Think scores higher on coding benchmarks: 37.7 versus 20.2 in the Noometry coding category.

How many benchmarks do Llama 4 Scout and Olmo 3.1 32b Think share?

15 benchmarks have published results for both models. Llama 4 Scout has 43 scored results on Noometry and Olmo 3.1 32b Think has 15.

Related comparisons

Go deeper