Noometry

Model comparison

Gemini 1.5 Flash 8B vs Pixtral Large

Pixtral Large is the stronger model overall, scoring 32.2 to 29.9 on the Noometry Index.

Last verified . 1 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.9

Rank #301 Confirmed

Pixtral Large Mistral AI

32.2

Rank #259 Reported

Summary

  • They share 1 benchmark with published results for both. Gemini 1.5 Flash 8B scores higher in 1 category and Pixtral Large in 2 categories; 3 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Gemini 1.5 Flash 8B leads 42.8 to 32.9.
  • Pixtral Large has downloadable open weights; the other is API-only.

Side by side

Gemini 1.5 Flash 8BPixtral Large
ProviderGoogleMistral AI
Noometry Index29.932.2
Released2024-10-032024-11-01
WeightsProprietaryOpen
Context window—128K
Max output—128K
Input $ / M tokens—$2
Output $ / M tokens—$6
Results tracked213

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Gemini 1.5 Flash 8B: 35.5 (#225), Pixtral Large: —

BenchmarkGemini 1.5 Flash 8BPixtral Large
LMArena Coding1218—

Reasoning Pixtral Large leads

Gemini 1.5 Flash 8B: 20.0 (#244), Pixtral Large: 21.7 (#218)

BenchmarkGemini 1.5 Flash 8BPixtral Large
EnigmaEval—0.8%
LMArena Hard Prompts1209—
DTBench50%—

Math Not comparable

Gemini 1.5 Flash 8B: 14.2 (#302), Pixtral Large: —

BenchmarkGemini 1.5 Flash 8BPixtral Large
OTIS Mock AIME 2024-20254.6%—
LMArena Math1207—

Knowledge Not comparable

Gemini 1.5 Flash 8B: 16.0 (#289), Pixtral Large: —

BenchmarkGemini 1.5 Flash 8BPixtral Large
GPQA Diamond33%—
LMArena Expert1185—

Multimodal Pixtral Large leads

Gemini 1.5 Flash 8B: 28.2 (#115), Pixtral Large: 30.6 (#111)

BenchmarkGemini 1.5 Flash 8BPixtral Large
LMArena Vision10441089

Multilingual Not comparable

Gemini 1.5 Flash 8B: 38.5 (#229), Pixtral Large: —

BenchmarkGemini 1.5 Flash 8BPixtral Large
LMArena Non-English1215—
LMArena Chinese1231—
LMArena French1234—
LMArena German1206—
LMArena Japanese1150—
LMArena Korean1140—
LMArena Russian1236—
LMArena Spanish1212—

Instruction Following Not comparable

Gemini 1.5 Flash 8B: 62.8 (#236), Pixtral Large: —

BenchmarkGemini 1.5 Flash 8BPixtral Large
LMArena Instruction Following1199—

Long Context Not comparable

Gemini 1.5 Flash 8B: 37.0 (#225), Pixtral Large: —

BenchmarkGemini 1.5 Flash 8BPixtral Large
LMArena Longer Query1219—

Writing & Preference Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 42.8 (#232), Pixtral Large: 32.9 (#278)

BenchmarkGemini 1.5 Flash 8BPixtral Large
LMArena Text1226—
LMArena Creative Writing1218—
EQ-Bench Creative Writing—988
LMArena Multi-Turn1185—

Frequently asked questions

Is Gemini 1.5 Flash 8B better than Pixtral Large?

Pixtral Large is the stronger model overall, scoring 32.2 to 29.9 on the Noometry Index.

How many benchmarks do Gemini 1.5 Flash 8B and Pixtral Large share?

1 benchmark has published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and Pixtral Large has 3.

Related comparisons

Go deeper