Noometry

Model comparison

Mistral Medium vs Olmo 3 32b Think

Olmo 3 32b Think is the stronger model overall, scoring 38.7 to 36.3 on the Noometry Index.

Last verified . 14 shared benchmarks.

Mistral Medium Mistral AI

36.3

Rank #218 Confirmed

Olmo 3 32b Think Allen Institute for AI (Ai2)

38.7

Rank #183 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Mistral Medium scores higher in 4 categories and Olmo 3 32b Think in 4 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in multilingual, where Mistral Medium leads 52.1 to 41.2.

Side by side

Mistral MediumOlmo 3 32b Think
ProviderMistral AIAllen Institute for AI (Ai2)
Noometry Index36.338.7
Released2023-12-11—
WeightsOpenOpen
Context window262K—
Max output262K—
Input $ / M tokens$1.50—
Output $ / M tokens$7.50—
Results tracked3614

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Olmo 3 32b Think leads

Mistral Medium: 34.2 (#243), Olmo 3 32b Think: 38.6 (#172)

BenchmarkMistral MediumOlmo 3 32b Think
LMArena Coding14341319
FrontierCode8%—
SciCode40.2%—
WeirdML43.7%—
ALE-Bench763.98—

Agentic & Tool Use Not comparable

Mistral Medium: 28.3 (#90), Olmo 3 32b Think: —

BenchmarkMistral MediumOlmo 3 32b Think
Berkeley Function Calling Leaderboard37.7%—

Reasoning Olmo 3 32b Think leads

Mistral Medium: 24.0 (#167), Olmo 3 32b Think: 25.9 (#140)

BenchmarkMistral MediumOlmo 3 32b Think
LMArena Hard Prompts14261302
Kagi LLM Benchmark50%—
CritPt0%—
DTBench75.5%—
LMCA26.1%—
Surface Evolver Bench26.9%—

Math Olmo 3 32b Think leads

Mistral Medium: 28.1 (#245), Olmo 3 32b Think: 36.5 (#165)

BenchmarkMistral MediumOlmo 3 32b Think
LMArena Math14081316
OTIS Mock AIME 2024-202532.2%—
ProofBench9%—
MATH Level 581.6%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge Olmo 3 32b Think leads

Mistral Medium: 25.0 (#265), Olmo 3 32b Think: 35.0 (#190)

BenchmarkMistral MediumOlmo 3 32b Think
LMArena Expert14081273
GPQA Diamond59.5%—
Humanity's Last Exam4.5%—
Vectara Hallucination Rate22.7%—

Multimodal Not comparable

Mistral Medium: 35.3 (#88), Olmo 3 32b Think: —

BenchmarkMistral MediumOlmo 3 32b Think
LMArena Vision1172—

Multilingual Mistral Medium leads

Mistral Medium: 52.1 (#91), Olmo 3 32b Think: 41.2 (#210)

BenchmarkMistral MediumOlmo 3 32b Think
LMArena Non-English14081255
LMArena Chinese14471300
LMArena French14591291
LMArena German14321290
LMArena Russian14111254
LMArena Japanese1378—
LMArena Korean1380—
LMArena Spanish1433—

Instruction Following Mistral Medium leads

Mistral Medium: 73.7 (#116), Olmo 3 32b Think: 67.2 (#198)

BenchmarkMistral MediumOlmo 3 32b Think
LMArena Instruction Following13981275

Long Context Mistral Medium leads

Mistral Medium: 42.9 (#114), Olmo 3 32b Think: 39.4 (#182)

BenchmarkMistral MediumOlmo 3 32b Think
LMArena Longer Query14061296

Writing & Preference Mistral Medium leads

Mistral Medium: 60.0 (#103), Olmo 3 32b Think: 49.1 (#193)

BenchmarkMistral MediumOlmo 3 32b Think
LMArena Text14241300
LMArena Creative Writing13911256
LMArena Multi-Turn14181290
Short-Story Creative Writing77.3%—

Frequently asked questions

Is Mistral Medium better than Olmo 3 32b Think?

Olmo 3 32b Think is the stronger model overall, scoring 38.7 to 36.3 on the Noometry Index.

Is Mistral Medium or Olmo 3 32b Think better for coding?

Olmo 3 32b Think scores higher on coding benchmarks: 38.6 versus 34.2 in the Noometry coding category.

How many benchmarks do Mistral Medium and Olmo 3 32b Think share?

14 benchmarks have published results for both models. Mistral Medium has 36 scored results on Noometry and Olmo 3 32b Think has 14.

Related comparisons

Go deeper