Noometry

Model comparison

Gemini 1.5 Flash 8B vs Mistral Small 3.1

Mistral Small 3.1 is the stronger model overall, scoring 31.4 to 29.7 on the Noometry Index.

Last verified . 20 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.7

Rank #292 Confirmed

Mistral Small 3.1 Mistral AI

31.4

Rank #263 Confirmed

Summary

  • They share 20 benchmarks with published results for both. Gemini 1.5 Flash 8B scores higher in 2 categories and Mistral Small 3.1 in 7 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Mistral Small 3.1 leads 22.6 to 16.0.
  • The biggest single-benchmark swing is GPQA Diamond: 33% for Gemini 1.5 Flash 8B and 41.9% for Mistral Small 3.1.
  • Mistral Small 3.1 has downloadable open weights; the other is API-only.

Side by side

Gemini 1.5 Flash 8BMistral Small 3.1
ProviderGoogleMistral AI
Noometry Index29.731.4
Released2024-10-032025-03-17
WeightsProprietaryOpen
Context window—128K
Max output—102K
Input $ / M tokens—$0.35
Output $ / M tokens—$0.56
Results tracked2128

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Small 3.1 leads

Gemini 1.5 Flash 8B: 34.9 (#221), Mistral Small 3.1: 37.7 (#175)

BenchmarkGemini 1.5 Flash 8BMistral Small 3.1
LMArena Coding12181309

Reasoning Too close to call

Gemini 1.5 Flash 8B: 19.2 (#242), Mistral Small 3.1: 18.9 (#249)

BenchmarkGemini 1.5 Flash 8BMistral Small 3.1
LMArena Hard Prompts12101278
Chess Puzzles—1%
DTBench50%—
Epoch Capabilities Index—127.48

Math Too close to call

Gemini 1.5 Flash 8B: 13.3 (#296), Mistral Small 3.1: 13.7 (#295)

BenchmarkGemini 1.5 Flash 8BMistral Small 3.1
OTIS Mock AIME 2024-20254.6%3.9%
LMArena Math12071261
Omni-MATH—24.8%

Knowledge Mistral Small 3.1 leads

Gemini 1.5 Flash 8B: 16.0 (#285), Mistral Small 3.1: 22.6 (#266)

BenchmarkGemini 1.5 Flash 8BMistral Small 3.1
GPQA Diamond33%41.9%
LMArena Expert11851258
MMLU-Pro—61%
GPQA (HELM)—39.2%

Multimodal Mistral Small 3.1 leads

Gemini 1.5 Flash 8B: 27.6 (#110), Mistral Small 3.1: 32.6 (#93)

BenchmarkGemini 1.5 Flash 8BMistral Small 3.1
LMArena Vision10441136

Multilingual Mistral Small 3.1 leads

Gemini 1.5 Flash 8B: 38.6 (#224), Mistral Small 3.1: 41.4 (#204)

BenchmarkGemini 1.5 Flash 8BMistral Small 3.1
LMArena Non-English12151255
LMArena Chinese12311254
LMArena French12331270
LMArena German12061266
LMArena Japanese11501210
LMArena Korean11401206
LMArena Russian12361263
LMArena Spanish12121283

Instruction Following Too close to call

Gemini 1.5 Flash 8B: 63.1 (#231), Mistral Small 3.1: 63.8 (#225)

BenchmarkGemini 1.5 Flash 8BMistral Small 3.1
LMArena Instruction Following11991264
IFEval—75%

Long Context Mistral Small 3.1 leads

Gemini 1.5 Flash 8B: 36.9 (#220), Mistral Small 3.1: 39.5 (#172)

BenchmarkGemini 1.5 Flash 8BMistral Small 3.1
LMArena Longer Query12191300

Writing & Preference Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 43.0 (#227), Mistral Small 3.1: 37.2 (#254)

BenchmarkGemini 1.5 Flash 8BMistral Small 3.1
LMArena Text12261278
LMArena Creative Writing12181253
LMArena Multi-Turn11851270
EQ-Bench Creative Writing—761
WildBench—78.8%

Frequently asked questions

Is Gemini 1.5 Flash 8B better than Mistral Small 3.1?

Mistral Small 3.1 is the stronger model overall, scoring 31.4 to 29.7 on the Noometry Index.

Is Gemini 1.5 Flash 8B or Mistral Small 3.1 better for coding?

Mistral Small 3.1 scores higher on coding benchmarks: 37.7 versus 34.9 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash 8B and Mistral Small 3.1 share?

20 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and Mistral Small 3.1 has 28.

Related comparisons

Go deeper