Noometry

Model comparison

Mistral Medium vs Olmo 3.1 32b Think

Olmo 3.1 32b Think is the stronger model overall, scoring 37.3 to 36.0 on the Noometry Index.

Last verified . 15 shared benchmarks.

Mistral Medium Mistral AI

36.0

Rank #211 Confirmed

Olmo 3.1 32b Think Allen Institute for AI (Ai2)

37.3

Rank #191 Confirmed

Summary

  • They share 15 benchmarks with published results for both. Mistral Medium scores higher in 4 categories and Olmo 3.1 32b Think in 4 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in multilingual, where Mistral Medium leads 52.3 to 38.3.

Side by side

Mistral MediumOlmo 3.1 32b Think
ProviderMistral AIAllen Institute for AI (Ai2)
Noometry Index36.037.3
Released2023-12-11—
WeightsOpenOpen
Context window262K—
Max output262K—
Input $ / M tokens$1.50—
Output $ / M tokens$7.50—
Results tracked3615

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Olmo 3.1 32b Think leads

Mistral Medium: 33.8 (#241), Olmo 3.1 32b Think: 37.1 (#186)

BenchmarkMistral MediumOlmo 3.1 32b Think
LMArena Coding14341292
FrontierCode8%—
SciCode40.2%—
WeirdML43.7%—
ALE-Bench763.98—

Agentic & Tool Use Not comparable

Mistral Medium: 28.1 (#84), Olmo 3.1 32b Think: —

BenchmarkMistral MediumOlmo 3.1 32b Think
Berkeley Function Calling Leaderboard37.7%—

Reasoning Too close to call

Mistral Medium: 23.3 (#165), Olmo 3.1 32b Think: 24.1 (#151)

BenchmarkMistral MediumOlmo 3.1 32b Think
LMArena Hard Prompts14261272
Kagi LLM Benchmark50%—
CritPt0%—
DTBench75.5%—
LMCA26.1%—
Surface Evolver Bench26.9%—

Math Olmo 3.1 32b Think leads

Mistral Medium: 27.2 (#238), Olmo 3.1 32b Think: 33.6 (#170)

BenchmarkMistral MediumOlmo 3.1 32b Think
LMArena Math14091305
OTIS Mock AIME 2024-202532.2%—
ProofBench9%—
MATH Level 581.6%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge Olmo 3.1 32b Think leads

Mistral Medium: 24.9 (#260), Olmo 3.1 32b Think: 35.0 (#178)

BenchmarkMistral MediumOlmo 3.1 32b Think
LMArena Expert14081294
GPQA Diamond59.5%—
Humanity's Last Exam4.5%—
Vectara Hallucination Rate22.7%—

Multimodal Not comparable

Mistral Medium: 34.6 (#82), Olmo 3.1 32b Think: —

BenchmarkMistral MediumOlmo 3.1 32b Think
LMArena Vision1172—

Multilingual Mistral Medium leads

Mistral Medium: 52.3 (#86), Olmo 3.1 32b Think: 38.3 (#226)

BenchmarkMistral MediumOlmo 3.1 32b Think
LMArena Non-English14081210
LMArena Chinese14481241
LMArena French14601262
LMArena German14331260
LMArena Russian14111193
LMArena Spanish14331288
LMArena Japanese1378—
LMArena Korean1382—

Instruction Following Mistral Medium leads

Mistral Medium: 73.9 (#111), Olmo 3.1 32b Think: 65.8 (#213)

BenchmarkMistral MediumOlmo 3.1 32b Think
LMArena Instruction Following13981246

Long Context Mistral Medium leads

Mistral Medium: 42.9 (#108), Olmo 3.1 32b Think: 38.6 (#190)

BenchmarkMistral MediumOlmo 3.1 32b Think
LMArena Longer Query14061272

Writing & Preference Mistral Medium leads

Mistral Medium: 60.2 (#97), Olmo 3.1 32b Think: 46.4 (#215)

BenchmarkMistral MediumOlmo 3.1 32b Think
LMArena Text14241271
LMArena Creative Writing13911226
LMArena Multi-Turn14181252
Short-Story Creative Writing77.3%—

Frequently asked questions

Is Mistral Medium better than Olmo 3.1 32b Think?

Olmo 3.1 32b Think is the stronger model overall, scoring 37.3 to 36.0 on the Noometry Index.

Is Mistral Medium or Olmo 3.1 32b Think better for coding?

Olmo 3.1 32b Think scores higher on coding benchmarks: 37.1 versus 33.8 in the Noometry coding category.

How many benchmarks do Mistral Medium and Olmo 3.1 32b Think share?

15 benchmarks have published results for both models. Mistral Medium has 36 scored results on Noometry and Olmo 3.1 32b Think has 15.

Related comparisons

Go deeper