Noometry

Model comparison

Mistral Small 3.1 vs Olmo 2 0325 32b Instruct

Olmo 2 0325 32b Instruct is the stronger model overall, scoring 32.7 to 31.7 on the Noometry Index.

Last verified . 16 shared benchmarks.

Mistral Small 3.1 Mistral AI

31.7

Rank #269 Confirmed

Olmo 2 0325 32b Instruct Allen Institute for AI (Ai2)

32.7

Rank #254 Confirmed

Summary

  • They share 16 benchmarks with published results for both. Mistral Small 3.1 scores higher in 5 categories and Olmo 2 0325 32b Instruct in 3 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Olmo 2 0325 32b Instruct leads 26.8 to 14.7.
  • The biggest single-benchmark swing is MMLU-Pro: 61% for Mistral Small 3.1 and 41.4% for Olmo 2 0325 32b Instruct.

Side by side

Mistral Small 3.1Olmo 2 0325 32b Instruct
ProviderMistral AIAllen Institute for AI (Ai2)
Noometry Index31.732.7
Released2025-03-17—
WeightsOpenOpen
Context window128K—
Max output102K—
Input $ / M tokens$0.35—
Output $ / M tokens$0.56—
Results tracked2816

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Small 3.1 leads

Mistral Small 3.1: 38.3 (#179), Olmo 2 0325 32b Instruct: 35.2 (#227)

BenchmarkMistral Small 3.1Olmo 2 0325 32b Instruct
LMArena Coding13091210

Reasoning Olmo 2 0325 32b Instruct leads

Mistral Small 3.1: 19.7 (#254), Olmo 2 0325 32b Instruct: 23.6 (#175)

BenchmarkMistral Small 3.1Olmo 2 0325 32b Instruct
LMArena Hard Prompts12781208
Chess Puzzles1%—
Epoch Capabilities Index127.48—

Math Olmo 2 0325 32b Instruct leads

Mistral Small 3.1: 14.7 (#301), Olmo 2 0325 32b Instruct: 26.8 (#255)

BenchmarkMistral Small 3.1Olmo 2 0325 32b Instruct
Omni-MATH24.8%16.1%
LMArena Math12621208
OTIS Mock AIME 2024-20253.9%—

Knowledge Mistral Small 3.1 leads

Mistral Small 3.1: 22.6 (#271), Olmo 2 0325 32b Instruct: 19.5 (#279)

BenchmarkMistral Small 3.1Olmo 2 0325 32b Instruct
MMLU-Pro61%41.4%
GPQA (HELM)39.2%28.7%
GPQA Diamond41.9%—
LMArena Expert1257—

Multimodal Not comparable

Mistral Small 3.1: 33.2 (#99), Olmo 2 0325 32b Instruct: —

BenchmarkMistral Small 3.1Olmo 2 0325 32b Instruct
LMArena Vision1136—

Multilingual Mistral Small 3.1 leads

Mistral Small 3.1: 41.2 (#209), Olmo 2 0325 32b Instruct: 34.8 (#248)

BenchmarkMistral Small 3.1Olmo 2 0325 32b Instruct
LMArena Non-English12551160
LMArena Chinese12531192
LMArena Russian12631187
LMArena French1273—
LMArena German1266—
LMArena Japanese1208—
LMArena Korean1206—
LMArena Spanish1283—

Instruction Following Mistral Small 3.1 leads

Mistral Small 3.1: 63.6 (#230), Olmo 2 0325 32b Instruct: 61.5 (#244)

BenchmarkMistral Small 3.1Olmo 2 0325 32b Instruct
IFEval75%78%
LMArena Instruction Following12641186

Long Context Mistral Small 3.1 leads

Mistral Small 3.1: 39.5 (#178), Olmo 2 0325 32b Instruct: 36.2 (#234)

BenchmarkMistral Small 3.1Olmo 2 0325 32b Instruct
LMArena Longer Query12991194

Writing & Preference Olmo 2 0325 32b Instruct leads

Mistral Small 3.1: 37.0 (#259), Olmo 2 0325 32b Instruct: 42.1 (#236)

BenchmarkMistral Small 3.1Olmo 2 0325 32b Instruct
LMArena Text12771218
LMArena Creative Writing12531199
WildBench78.8%73.4%
LMArena Multi-Turn12701221
EQ-Bench Creative Writing761—

Frequently asked questions

Is Mistral Small 3.1 better than Olmo 2 0325 32b Instruct?

Olmo 2 0325 32b Instruct is the stronger model overall, scoring 32.7 to 31.7 on the Noometry Index.

Is Mistral Small 3.1 or Olmo 2 0325 32b Instruct better for coding?

Mistral Small 3.1 scores higher on coding benchmarks: 38.3 versus 35.2 in the Noometry coding category.

How many benchmarks do Mistral Small 3.1 and Olmo 2 0325 32b Instruct share?

16 benchmarks have published results for both models. Mistral Small 3.1 has 28 scored results on Noometry and Olmo 2 0325 32b Instruct has 16.

Related comparisons

Go deeper