Noometry

Model comparison

Gemini Advanced 0514 vs Mistral Small

Gemini Advanced 0514 is the stronger model overall, scoring 37.7 to 33.4 on the Noometry Index.

Last verified . 17 shared benchmarks.

Gemini Advanced 0514 Google

37.7

Rank #197 Confirmed

Mistral Small Mistral AI

33.4

Rank #243 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Gemini Advanced 0514 scores higher in 4 categories and Mistral Small in 4 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Gemini Advanced 0514 leads 35.5 to 16.4.
  • Mistral Small has downloadable open weights; the other is API-only.

Side by side

Gemini Advanced 0514Mistral Small
ProviderGoogleMistral AI
Noometry Index37.733.4
Released—2024-02-26
WeightsProprietaryOpen
Context window—262K
Max output—256K
Input $ / M tokens—$0.15
Output $ / M tokens—$0.60
Results tracked1739

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini Advanced 0514 leads

Gemini Advanced 0514: 36.6 (#205), Mistral Small: 34.0 (#247)

BenchmarkGemini Advanced 0514Mistral Small
LMArena Coding12551362
SciCode—26.5%
BigCodeBench Instruct—36.1%
LiveBench Coding—36.2%
BigCodeBench Complete—46.6%
ALE-Bench—497.62

Agentic & Tool Use Not comparable

Gemini Advanced 0514: —, Mistral Small: 28.1 (#93)

BenchmarkGemini Advanced 0514Mistral Small
Berkeley Function Calling Leaderboard—37.1%

Reasoning Gemini Advanced 0514 leads

Gemini Advanced 0514: 24.6 (#160), Mistral Small: 19.8 (#250)

BenchmarkGemini Advanced 0514Mistral Small
LMArena Hard Prompts12481335
Kagi LLM Benchmark—37.8%
CritPt—0%
LiveBench Reasoning—44.8%
DTBench—70.9%
LiveBench Data Analysis—53.7%
LMCA—20.6%
LiveBench—44%

Math Gemini Advanced 0514 leads

Gemini Advanced 0514: 35.5 (#181), Mistral Small: 16.4 (#293)

BenchmarkGemini Advanced 0514Mistral Small
LMArena Math12721341
OTIS Mock AIME 2024-2025—5.8%
LiveBench Math—39.9%
MATH Level 5—46.8%

Knowledge Gemini Advanced 0514 leads

Gemini Advanced 0514: 33.3 (#205), Mistral Small: 31.0 (#222)

BenchmarkGemini Advanced 0514Mistral Small
LMArena Expert12151291
GPQA Diamond—47.5%
Vectara Hallucination Rate—5.1%
MMLU—68.7%

Multimodal Not comparable

Gemini Advanced 0514: —, Mistral Small: 33.5 (#96)

BenchmarkGemini Advanced 0514Mistral Small
LMArena Vision—1142

Multilingual Mistral Small leads

Gemini Advanced 0514: 42.6 (#192), Mistral Small: 45.5 (#169)

BenchmarkGemini Advanced 0514Mistral Small
LMArena Non-English12741315
LMArena Chinese12741340
LMArena French13071337
LMArena German12751340
LMArena Japanese12421275
LMArena Korean12321259
LMArena Russian12831324
LMArena Spanish12421346

Instruction Following Too close to call

Gemini Advanced 0514: 66.0 (#212), Mistral Small: 66.4 (#209)

BenchmarkGemini Advanced 0514Mistral Small
LMArena Instruction Following12531310
LiveBench Instruction Following—63.7%

Long Context Mistral Small leads

Gemini Advanced 0514: 38.0 (#207), Mistral Small: 40.4 (#156)

BenchmarkGemini Advanced 0514Mistral Small
LMArena Longer Query12531327

Writing & Preference Mistral Small leads

Gemini Advanced 0514: 48.5 (#200), Mistral Small: 52.5 (#171)

BenchmarkGemini Advanced 0514Mistral Small
LMArena Text12791338
LMArena Creative Writing12841305
LMArena Multi-Turn12691344
LiveBench Language—30.5%

Frequently asked questions

Is Gemini Advanced 0514 better than Mistral Small?

Gemini Advanced 0514 is the stronger model overall, scoring 37.7 to 33.4 on the Noometry Index.

Is Gemini Advanced 0514 or Mistral Small better for coding?

Gemini Advanced 0514 scores higher on coding benchmarks: 36.6 versus 34.0 in the Noometry coding category.

How many benchmarks do Gemini Advanced 0514 and Mistral Small share?

17 benchmarks have published results for both models. Gemini Advanced 0514 has 17 scored results on Noometry and Mistral Small has 39.

Related comparisons

Go deeper