Noometry

Model comparison

Gemini 1.0 Pro vs Mistral Medium

Mistral Medium is the stronger model overall, scoring 36.3 to 27.3 on the Noometry Index.

Last verified . 20 shared benchmarks.

Gemini 1.0 Pro Google

27.3

Rank #332 Confirmed

Mistral Medium Mistral AI

36.3

Rank #218 Confirmed

Summary

  • They share 20 benchmarks with published results for both. Gemini 1.0 Pro scores higher in 0 categories and Mistral Medium in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Mistral Medium leads 60.0 to 36.0.
  • The biggest single-benchmark swing is MATH Level 5: 11.2% for Gemini 1.0 Pro and 81.6% for Mistral Medium.
  • Mistral Medium has downloadable open weights; the other is API-only.

Side by side

Gemini 1.0 ProMistral Medium
ProviderGoogleMistral AI
Noometry Index27.336.3
Released2023-12-132023-12-11
WeightsProprietaryOpen
Context window—262K
Max output—262K
Input $ / M tokens—$1.50
Output $ / M tokens—$7.50
Results tracked2436

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Medium leads

Gemini 1.0 Pro: 32.2 (#275), Mistral Medium: 34.2 (#243)

BenchmarkGemini 1.0 ProMistral Medium
LMArena Coding11081434
FrontierCode—8%
SciCode—40.2%
WeirdML—43.7%
ALE-Bench—763.98
HumanEval+55.5%—
MBPP+61.4%—

Agentic & Tool Use Not comparable

Gemini 1.0 Pro: —, Mistral Medium: 28.3 (#90)

BenchmarkGemini 1.0 ProMistral Medium
Berkeley Function Calling Leaderboard—37.7%

Reasoning Mistral Medium leads

Gemini 1.0 Pro: 17.1 (#296), Mistral Medium: 24.0 (#167)

BenchmarkGemini 1.0 ProMistral Medium
LMArena Hard Prompts11091426
DTBench45.9%75.5%
Kagi LLM Benchmark—50%
CritPt—0%
LMCA—26.1%
Surface Evolver Bench—26.9%
Epoch Capabilities Index117.04—

Math Mistral Medium leads

Gemini 1.0 Pro: 9.3 (#321), Mistral Medium: 28.1 (#245)

BenchmarkGemini 1.0 ProMistral Medium
OTIS Mock AIME 2024-20251.1%32.2%
LMArena Math11321408
MATH Level 511.2%81.6%
ProofBench—9%
FrontierMath (Feb 2025 set)—0.3%

Knowledge Mistral Medium leads

Gemini 1.0 Pro: 15.6 (#291), Mistral Medium: 25.0 (#265)

BenchmarkGemini 1.0 ProMistral Medium
GPQA Diamond34%59.5%
LMArena Expert10591408
Humanity's Last Exam—4.5%
Vectara Hallucination Rate—22.7%
MMLU70%—

Multimodal Not comparable

Gemini 1.0 Pro: —, Mistral Medium: 35.3 (#88)

BenchmarkGemini 1.0 ProMistral Medium
LMArena Vision—1172

Multilingual Mistral Medium leads

Gemini 1.0 Pro: 33.4 (#252), Mistral Medium: 52.1 (#91)

BenchmarkGemini 1.0 ProMistral Medium
LMArena Non-English11381408
LMArena Chinese11241447
LMArena French11451459
LMArena German11251432
LMArena Japanese10231378
LMArena Russian11861411
LMArena Spanish11191433
LMArena Korean—1380

Instruction Following Mistral Medium leads

Gemini 1.0 Pro: 57.6 (#267), Mistral Medium: 73.7 (#116)

BenchmarkGemini 1.0 ProMistral Medium
LMArena Instruction Following11141398

Long Context Mistral Medium leads

Gemini 1.0 Pro: 34.3 (#249), Mistral Medium: 42.9 (#114)

BenchmarkGemini 1.0 ProMistral Medium
LMArena Longer Query11321406

Writing & Preference Mistral Medium leads

Gemini 1.0 Pro: 36.0 (#264), Mistral Medium: 60.0 (#103)

BenchmarkGemini 1.0 ProMistral Medium
LMArena Text11491424
LMArena Creative Writing11311391
LMArena Multi-Turn11391418
Short-Story Creative Writing—77.3%

Frequently asked questions

Is Gemini 1.0 Pro better than Mistral Medium?

Mistral Medium is the stronger model overall, scoring 36.3 to 27.3 on the Noometry Index.

Is Gemini 1.0 Pro or Mistral Medium better for coding?

Mistral Medium scores higher on coding benchmarks: 34.2 versus 32.2 in the Noometry coding category.

How many benchmarks do Gemini 1.0 Pro and Mistral Medium share?

20 benchmarks have published results for both models. Gemini 1.0 Pro has 24 scored results on Noometry and Mistral Medium has 36.

Related comparisons

Go deeper