Model comparison

Gemini 1.0 Pro vs Mercury 2

Mercury 2 is the stronger model overall, scoring 39.1 to 27.3 on the Noometry Index.

Last verified . 11 shared benchmarks.

Gemini 1.0 Pro Google

27.3

Rank #332 Confirmed

Mercury 2 Inception

39.1

Rank #175 Confirmed

Summary

  • They share 11 benchmarks with published results for both. Gemini 1.0 Pro scores higher in 0 categories and Mercury 2 in 7 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Mercury 2 leads 36.2 to 15.6.

Side by side

Gemini 1.0 Pro and Mercury 2 specifications
Gemini 1.0 ProMercury 2
ProviderGoogleInception
Noometry Index27.339.1
Released2023-12-132026-02-20
WeightsProprietaryProprietary
Context window—128K
Max output—50K
Input $ / M tokens—$0.25
Output $ / M tokens—$0.75
Results tracked2417

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mercury 2 leads

Gemini 1.0 Pro: 32.2 (#275), Mercury 2: 33.5 (#255)

Coding benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Coding11081391
LMArena WebDev—1171
SciCode—38.7%
WeirdML—43.2%
ALE-Bench—785.58
HumanEval+55.5%—
MBPP+61.4%—

Reasoning Mercury 2 leads

Gemini 1.0 Pro: 17.1 (#296), Mercury 2: 23.8 (#170)

Reasoning benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Hard Prompts11091362
CritPt—0.8%
DTBench45.9%—
Epoch Capabilities Index117.04—

Math Not comparable

Gemini 1.0 Pro: 9.3 (#321), Mercury 2: —

Math benchmarks
BenchmarkGemini 1.0 ProMercury 2
OTIS Mock AIME 2024-20251.1%—
LMArena Math1132—
MATH Level 511.2%—

Knowledge Mercury 2 leads

Gemini 1.0 Pro: 15.6 (#291), Mercury 2: 36.2 (#172)

Knowledge benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Expert10591358
GPQA Diamond34%—
Vectara Hallucination Rate—12.3%
MMLU70%—

Multilingual Mercury 2 leads

Gemini 1.0 Pro: 33.4 (#252), Mercury 2: 46.6 (#157)

Multilingual benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Non-English11381331
LMArena Chinese11241417
LMArena Russian11861304
LMArena French1145—
LMArena German1125—
LMArena Japanese1023—
LMArena Spanish1119—

Instruction Following Mercury 2 leads

Gemini 1.0 Pro: 57.6 (#267), Mercury 2: 70.2 (#165)

Instruction Following benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Instruction Following11141329

Long Context Mercury 2 leads

Gemini 1.0 Pro: 34.3 (#249), Mercury 2: 40.5 (#154)

Long Context benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Longer Query11321330

Writing & Preference Mercury 2 leads

Gemini 1.0 Pro: 36.0 (#264), Mercury 2: 53.8 (#155)

Writing & Preference benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Text11491355
LMArena Creative Writing11311289
LMArena Multi-Turn11391358

Frequently asked questions

Is Gemini 1.0 Pro better than Mercury 2?

Mercury 2 is the stronger model overall, scoring 39.1 to 27.3 on the Noometry Index.

Is Gemini 1.0 Pro or Mercury 2 better for coding?

Mercury 2 scores higher on coding benchmarks: 33.5 versus 32.2 in the Noometry coding category.

How many benchmarks do Gemini 1.0 Pro and Mercury 2 share?

11 benchmarks have published results for both models. Gemini 1.0 Pro has 24 scored results on Noometry and Mercury 2 has 17.

Related comparisons

Go deeper