Noometry

Model comparison

Gemini Advanced 0514 vs Mistral Medium

Gemini Advanced 0514 is the stronger model overall, scoring 37.1 to 36.0 on the Noometry Index.

Last verified . 17 shared benchmarks.

Gemini Advanced 0514 Google

37.1

Rank #195 Confirmed

Mistral Medium Mistral AI

36.0

Rank #211 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Gemini Advanced 0514 scores higher in 4 categories and Mistral Medium in 4 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Mistral Medium leads 60.2 to 48.8.
  • Mistral Medium has downloadable open weights; the other is API-only.

Side by side

Gemini Advanced 0514Mistral Medium
ProviderGoogleMistral AI
Noometry Index37.136.0
Released—2023-12-11
WeightsProprietaryOpen
Context window—262K
Max output—262K
Input $ / M tokens—$1.50
Output $ / M tokens—$7.50
Results tracked1736

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini Advanced 0514 leads

Gemini Advanced 0514: 36.0 (#202), Mistral Medium: 33.8 (#241)

BenchmarkGemini Advanced 0514Mistral Medium
LMArena Coding12551434
FrontierCode—8%
SciCode—40.2%
WeirdML—43.7%
ALE-Bench—763.98

Agentic & Tool Use Not comparable

Gemini Advanced 0514: —, Mistral Medium: 28.1 (#84)

BenchmarkGemini Advanced 0514Mistral Medium
Berkeley Function Calling Leaderboard—37.7%

Reasoning Too close to call

Gemini Advanced 0514: 23.6 (#163), Mistral Medium: 23.3 (#165)

BenchmarkGemini Advanced 0514Mistral Medium
LMArena Hard Prompts12481426
Kagi LLM Benchmark—50%
CritPt—0%
DTBench—75.5%
LMCA—26.1%
Surface Evolver Bench—26.9%

Math Gemini Advanced 0514 leads

Gemini Advanced 0514: 32.9 (#181), Mistral Medium: 27.2 (#238)

BenchmarkGemini Advanced 0514Mistral Medium
LMArena Math12721409
OTIS Mock AIME 2024-2025—32.2%
ProofBench—9%
MATH Level 5—81.6%
FrontierMath (Feb 2025 set)—0.3%

Knowledge Gemini Advanced 0514 leads

Gemini Advanced 0514: 32.7 (#201), Mistral Medium: 24.9 (#260)

BenchmarkGemini Advanced 0514Mistral Medium
LMArena Expert12151408
GPQA Diamond—59.5%
Humanity's Last Exam—4.5%
Vectara Hallucination Rate—22.7%

Multimodal Not comparable

Gemini Advanced 0514: —, Mistral Medium: 34.6 (#82)

BenchmarkGemini Advanced 0514Mistral Medium
LMArena Vision—1172

Multilingual Mistral Medium leads

Gemini Advanced 0514: 42.8 (#187), Mistral Medium: 52.3 (#86)

BenchmarkGemini Advanced 0514Mistral Medium
LMArena Non-English12741408
LMArena Chinese12741448
LMArena French13061460
LMArena German12751433
LMArena Japanese12421378
LMArena Korean12321382
LMArena Russian12831411
LMArena Spanish12421433

Instruction Following Mistral Medium leads

Gemini Advanced 0514: 66.2 (#206), Mistral Medium: 73.9 (#111)

BenchmarkGemini Advanced 0514Mistral Medium
LMArena Instruction Following12541398

Long Context Mistral Medium leads

Gemini Advanced 0514: 38.0 (#202), Mistral Medium: 42.9 (#108)

BenchmarkGemini Advanced 0514Mistral Medium
LMArena Longer Query12541406

Writing & Preference Mistral Medium leads

Gemini Advanced 0514: 48.8 (#195), Mistral Medium: 60.2 (#97)

BenchmarkGemini Advanced 0514Mistral Medium
LMArena Text12791424
LMArena Creative Writing12841391
LMArena Multi-Turn12691418
Short-Story Creative Writing—77.3%

Frequently asked questions

Is Gemini Advanced 0514 better than Mistral Medium?

Gemini Advanced 0514 is the stronger model overall, scoring 37.1 to 36.0 on the Noometry Index.

Is Gemini Advanced 0514 or Mistral Medium better for coding?

Gemini Advanced 0514 scores higher on coding benchmarks: 36.0 versus 33.8 in the Noometry coding category.

How many benchmarks do Gemini Advanced 0514 and Mistral Medium share?

17 benchmarks have published results for both models. Gemini Advanced 0514 has 17 scored results on Noometry and Mistral Medium has 36.

Related comparisons

Go deeper