Noometry

Model comparison

Gemini 1.5 Flash 8B vs Mistral Small 3

Mistral Small 3 is the stronger model overall, scoring 31.2 to 29.9 on the Noometry Index.

Last verified . 18 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.9

Rank #301 Confirmed

Mistral Small 3 Mistral AI

31.2

Rank #278 Confirmed

Summary

  • They share 18 benchmarks with published results for both. Gemini 1.5 Flash 8B scores higher in 3 categories and Mistral Small 3 in 5 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Gemini 1.5 Flash 8B leads 42.8 to 32.2.
  • The biggest single-benchmark swing is GPQA Diamond: 33% for Gemini 1.5 Flash 8B and 47.3% for Mistral Small 3.
  • Mistral Small 3 has downloadable open weights; the other is API-only.

Side by side

Gemini 1.5 Flash 8BMistral Small 3
ProviderGoogleMistral AI
Noometry Index29.931.2
Released2024-10-032025-01-30
WeightsProprietaryOpen
Context window—33K
Max output—16K
Input $ / M tokens—$0.05
Output $ / M tokens—$0.08
Results tracked2124

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Small 3 leads

Gemini 1.5 Flash 8B: 35.5 (#225), Mistral Small 3: 36.5 (#207)

BenchmarkGemini 1.5 Flash 8BMistral Small 3
LMArena Coding12181246
BigCodeBench Instruct—45.3%
BigCodeBench Complete—50.4%

Reasoning Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 20.0 (#244), Mistral Small 3: 18.9 (#273)

BenchmarkGemini 1.5 Flash 8BMistral Small 3
LMArena Hard Prompts12091233
Chess Puzzles—0%
DTBench50%—
Epoch Capabilities Index—127.07

Math Mistral Small 3 leads

Gemini 1.5 Flash 8B: 14.2 (#302), Mistral Small 3: 16.3 (#295)

BenchmarkGemini 1.5 Flash 8BMistral Small 3
OTIS Mock AIME 2024-20254.6%6.7%
LMArena Math12071240

Knowledge Mistral Small 3 leads

Gemini 1.5 Flash 8B: 16.0 (#289), Mistral Small 3: 25.1 (#263)

BenchmarkGemini 1.5 Flash 8BMistral Small 3
GPQA Diamond33%47.3%
LMArena Expert11851202
Confabulations—25.2%

Multimodal Not comparable

Gemini 1.5 Flash 8B: 28.2 (#115), Mistral Small 3: —

BenchmarkGemini 1.5 Flash 8BMistral Small 3
LMArena Vision1044—

Multilingual Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 38.5 (#229), Mistral Small 3: 37.3 (#236)

BenchmarkGemini 1.5 Flash 8BMistral Small 3
LMArena Non-English12151198
LMArena Chinese12311204
LMArena French12341203
LMArena German12061211
LMArena Japanese11501111
LMArena Korean11401188
LMArena Russian12361216
LMArena Spanish1212—

Instruction Following Too close to call

Gemini 1.5 Flash 8B: 62.8 (#236), Mistral Small 3: 63.7 (#229)

BenchmarkGemini 1.5 Flash 8BMistral Small 3
LMArena Instruction Following11991214

Long Context Too close to call

Gemini 1.5 Flash 8B: 37.0 (#225), Mistral Small 3: 37.8 (#211)

BenchmarkGemini 1.5 Flash 8BMistral Small 3
LMArena Longer Query12191246

Writing & Preference Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 42.8 (#232), Mistral Small 3: 32.2 (#280)

BenchmarkGemini 1.5 Flash 8BMistral Small 3
LMArena Text12261234
LMArena Creative Writing12181195
LMArena Multi-Turn11851217
EQ-Bench Creative Writing—707

Frequently asked questions

Is Gemini 1.5 Flash 8B better than Mistral Small 3?

Mistral Small 3 is the stronger model overall, scoring 31.2 to 29.9 on the Noometry Index.

Is Gemini 1.5 Flash 8B or Mistral Small 3 better for coding?

Mistral Small 3 scores higher on coding benchmarks: 36.5 versus 35.5 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash 8B and Mistral Small 3 share?

18 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and Mistral Small 3 has 24.

Related comparisons

Go deeper