Noometry

Model comparison

Mistral Small 3 vs Yi-1.5-34B

Mistral Small 3 and Yi-1.5-34B score almost the same on the Noometry Index (30.9 vs 30.2), so choose on price, context window or the category you care about most.

Last verified . 19 shared benchmarks.

Mistral Small 3 Mistral AI

30.9

Rank #268 Confirmed

Yi-1.5-34B 01.AI

30.2

Rank #286 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Mistral Small 3 scores higher in 5 categories and Yi-1.5-34B in 3 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Mistral Small 3 leads 25.0 to 14.8.
  • The biggest single-benchmark swing is GPQA Diamond: 47.3% for Mistral Small 3 and 32% for Yi-1.5-34B.

Side by side

Mistral Small 3Yi-1.5-34B
ProviderMistral AI01.AI
Noometry Index30.930.2
Released2025-01-302024-05-13
WeightsOpenOpen
Context window33K—
Max output16K—
Input $ / M tokens$0.05—
Output $ / M tokens$0.08—
Results tracked2421

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Small 3 leads

Mistral Small 3: 36.0 (#203), Yi-1.5-34B: 31.9 (#267)

BenchmarkMistral Small 3Yi-1.5-34B
BigCodeBench Instruct45.3%33.9%
LMArena Coding12461169
BigCodeBench Complete50.4%43.8%

Reasoning Yi-1.5-34B leads

Mistral Small 3: 18.2 (#269), Yi-1.5-34B: 21.5 (#195)

BenchmarkMistral Small 3Yi-1.5-34B
LMArena Hard Prompts12331160
Chess Puzzles0%—
Epoch Capabilities Index127.06—

Math Yi-1.5-34B leads

Mistral Small 3: 15.3 (#290), Yi-1.5-34B: 25.3 (#248)

BenchmarkMistral Small 3Yi-1.5-34B
LMArena Math12401182
OTIS Mock AIME 2024-20256.7%—
MATH Level 5—25.5%

Knowledge Mistral Small 3 leads

Mistral Small 3: 25.0 (#258), Yi-1.5-34B: 14.8 (#290)

BenchmarkMistral Small 3Yi-1.5-34B
GPQA Diamond47.3%32%
LMArena Expert12021144
Confabulations25.2%—

Multilingual Mistral Small 3 leads

Mistral Small 3: 37.5 (#231), Yi-1.5-34B: 32.5 (#251)

BenchmarkMistral Small 3Yi-1.5-34B
LMArena Non-English11981121
LMArena Chinese12041213
LMArena French12021155
LMArena German12111111
LMArena Japanese11111021
LMArena Korean11881005
LMArena Russian12161091
LMArena Spanish—1120

Instruction Following Mistral Small 3 leads

Mistral Small 3: 63.9 (#224), Yi-1.5-34B: 59.4 (#252)

BenchmarkMistral Small 3Yi-1.5-34B
LMArena Instruction Following12141139

Long Context Mistral Small 3 leads

Mistral Small 3: 37.8 (#206), Yi-1.5-34B: 34.6 (#243)

BenchmarkMistral Small 3Yi-1.5-34B
LMArena Longer Query12461144

Writing & Preference Yi-1.5-34B leads

Mistral Small 3: 32.4 (#275), Yi-1.5-34B: 37.6 (#252)

BenchmarkMistral Small 3Yi-1.5-34B
LMArena Text12341173
LMArena Creative Writing11951135
LMArena Multi-Turn12171153
EQ-Bench Creative Writing707—

Frequently asked questions

Is Mistral Small 3 better than Yi-1.5-34B?

Mistral Small 3 and Yi-1.5-34B score almost the same on the Noometry Index (30.9 vs 30.2), so choose on price, context window or the category you care about most.

Is Mistral Small 3 or Yi-1.5-34B better for coding?

Mistral Small 3 scores higher on coding benchmarks: 36.0 versus 31.9 in the Noometry coding category.

How many benchmarks do Mistral Small 3 and Yi-1.5-34B share?

19 benchmarks have published results for both models. Mistral Small 3 has 24 scored results on Noometry and Yi-1.5-34B has 21.

Related comparisons

Go deeper