Noometry

Model comparison

Gemini 1.5 Flash 8B vs Mistral Medium 3.1

Mistral Medium 3.1 is the stronger model overall, scoring 31.5 to 29.7 on the Noometry Index.

Last verified . 0 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.7

Rank #292 Confirmed

Mistral Medium 3.1 Mistral AI

31.5

Rank #260 Reported

Summary

  • The widest gap is in writing & preference, where Mistral Medium 3.1 leads 55.6 to 43.0.

Side by side

Gemini 1.5 Flash 8BMistral Medium 3.1
ProviderGoogleMistral AI
Noometry Index29.731.5
Released2024-10-03—
WeightsProprietaryProprietary
Context window—131K
Max output—105K
Input $ / M tokens—$0.40
Output $ / M tokens—$2
Results tracked213

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Gemini 1.5 Flash 8B: 34.9 (#221), Mistral Medium 3.1: —

BenchmarkGemini 1.5 Flash 8BMistral Medium 3.1
LMArena Coding1218—

Reasoning Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 19.2 (#242), Mistral Medium 3.1: 10.1 (#336)

BenchmarkGemini 1.5 Flash 8BMistral Medium 3.1
NYT Connections (extended)—6.5%
Thematic Generalization—20.3%
LMArena Hard Prompts1210—
DTBench50%—

Math Not comparable

Gemini 1.5 Flash 8B: 13.3 (#296), Mistral Medium 3.1: —

BenchmarkGemini 1.5 Flash 8BMistral Medium 3.1
OTIS Mock AIME 2024-20254.6%—
LMArena Math1207—

Knowledge Not comparable

Gemini 1.5 Flash 8B: 16.0 (#285), Mistral Medium 3.1: —

BenchmarkGemini 1.5 Flash 8BMistral Medium 3.1
GPQA Diamond33%—
LMArena Expert1185—

Multimodal Not comparable

Gemini 1.5 Flash 8B: 27.6 (#110), Mistral Medium 3.1: —

BenchmarkGemini 1.5 Flash 8BMistral Medium 3.1
LMArena Vision1044—

Multilingual Not comparable

Gemini 1.5 Flash 8B: 38.6 (#224), Mistral Medium 3.1: —

BenchmarkGemini 1.5 Flash 8BMistral Medium 3.1
LMArena Non-English1215—
LMArena Chinese1231—
LMArena French1233—
LMArena German1206—
LMArena Japanese1150—
LMArena Korean1140—
LMArena Russian1236—
LMArena Spanish1212—

Instruction Following Not comparable

Gemini 1.5 Flash 8B: 63.1 (#231), Mistral Medium 3.1: —

BenchmarkGemini 1.5 Flash 8BMistral Medium 3.1
LMArena Instruction Following1199—

Long Context Not comparable

Gemini 1.5 Flash 8B: 36.9 (#220), Mistral Medium 3.1: —

BenchmarkGemini 1.5 Flash 8BMistral Medium 3.1
LMArena Longer Query1219—

Writing & Preference Mistral Medium 3.1 leads

Gemini 1.5 Flash 8B: 43.0 (#227), Mistral Medium 3.1: 55.6 (#140)

BenchmarkGemini 1.5 Flash 8BMistral Medium 3.1
LMArena Text1226—
LMArena Creative Writing1218—
EQ-Bench Creative Writing—1476
LMArena Multi-Turn1185—

Frequently asked questions

Is Gemini 1.5 Flash 8B better than Mistral Medium 3.1?

Mistral Medium 3.1 is the stronger model overall, scoring 31.5 to 29.7 on the Noometry Index.

How many benchmarks do Gemini 1.5 Flash 8B and Mistral Medium 3.1 share?

0 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and Mistral Medium 3.1 has 3.

Related comparisons

Go deeper