Noometry

Model comparison

Gemini 1.0 Pro vs o3-mini

o3-mini is the stronger model overall, scoring 36.7 to 27.1 on the Noometry Index.

Last verified . 21 shared benchmarks.

Gemini 1.0 Pro Google

27.1

Rank #327 Confirmed

o3-mini OpenAI

36.7

Rank #205 Confirmed

Summary

  • They share 21 benchmarks with published results for both. Gemini 1.0 Pro scores higher in 2 categories and o3-mini in 6 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where o3-mini leads 38.3 to 15.5.
  • The biggest single-benchmark swing is MATH Level 5: 11.2% for Gemini 1.0 Pro and 96.5% for o3-mini.

Side by side

Gemini 1.0 Proo3-mini
ProviderGoogleOpenAI
Noometry Index27.136.7
Released2023-12-132024-12-20
WeightsProprietaryProprietary
Context window—200K
Max output—100K
Input $ / M tokens—$1.10
Output $ / M tokens—$4.40
Results tracked2451

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3-mini leads

Gemini 1.0 Pro: 31.6 (#270), o3-mini: 41.4 (#116)

BenchmarkGemini 1.0 Proo3-mini
LMArena Coding11081378
Aider Polyglot—60.4%
SciCode—39.8%
GSO—1.3%
WeirdML—43.7%
LiveBench Coding—82.7%
CadEval—54%
HumanEval+55.5%—
MBPP+61.4%—

Agentic & Tool Use Not comparable

Gemini 1.0 Pro: —, o3-mini: 29.3 (#79)

BenchmarkGemini 1.0 Proo3-mini
Cybench—22.5%

Reasoning Too close to call

Gemini 1.0 Pro: 16.4 (#292), o3-mini: 16.3 (#295)

BenchmarkGemini 1.0 Proo3-mini
LMArena Hard Prompts11091366
DTBench45.9%68.8%
Epoch Capabilities Index116.97140.35
ARC-AGI-2—3%
SimpleBench—22.8%
ARC-AGI-1—34.5%
CritPt—0.3%
Chess Puzzles—17%
LiveBench Reasoning—89.6%
Mystery Game Puzzles—7%
LiveBench Data Analysis—70.6%
LMCA—19%
ForecastBench—59.6
LiveBench—75.9%

Math o3-mini leads

Gemini 1.0 Pro: 8.7 (#315), o3-mini: 27.6 (#236)

BenchmarkGemini 1.0 Proo3-mini
OTIS Mock AIME 2024-20251.1%76.9%
LMArena Math11321396
MATH Level 511.2%96.5%
FrontierMath (Tiers 1-3)—18.6%
FrontierMath Tier 4—0%
LiveBench Math—77.3%
FrontierMath (Feb 2025 set)—12.4%
FrontierMath Tier 4 (v1)—4.2%

Knowledge o3-mini leads

Gemini 1.0 Pro: 15.5 (#286), o3-mini: 38.3 (#135)

BenchmarkGemini 1.0 Proo3-mini
GPQA Diamond34%77%
LMArena Expert10591364
SimpleQA Verified—15.3%
Confabulations—17.9%
MMLU70%—

Multilingual o3-mini leads

Gemini 1.0 Pro: 33.5 (#247), o3-mini: 45.9 (#159)

BenchmarkGemini 1.0 Proo3-mini
LMArena Non-English11381319
LMArena Chinese11241379
LMArena French11441333
LMArena German11251303
LMArena Japanese10231286
LMArena Russian11861304
LMArena Spanish11181321
LMArena Korean—1314

Instruction Following o3-mini leads

Gemini 1.0 Pro: 57.9 (#262), o3-mini: 75.3 (#67)

BenchmarkGemini 1.0 Proo3-mini
LMArena Instruction Following11141338
LiveBench Instruction Following—84.4%

Long Context Too close to call

Gemini 1.0 Pro: 34.3 (#244), o3-mini: 33.8 (#251)

BenchmarkGemini 1.0 Proo3-mini
LMArena Longer Query11321343
Fiction.LiveBench—50%

Writing & Preference o3-mini leads

Gemini 1.0 Pro: 36.2 (#259), o3-mini: 50.4 (#177)

BenchmarkGemini 1.0 Proo3-mini
LMArena Text11491337
LMArena Creative Writing11301286
LMArena Multi-Turn11391320
Short-Story Creative Writing—61.7%
LiveBench Language—50.7%

Frequently asked questions

Is Gemini 1.0 Pro better than o3-mini?

o3-mini is the stronger model overall, scoring 36.7 to 27.1 on the Noometry Index.

Is Gemini 1.0 Pro or o3-mini better for coding?

o3-mini scores higher on coding benchmarks: 41.4 versus 31.6 in the Noometry coding category.

How many benchmarks do Gemini 1.0 Pro and o3-mini share?

21 benchmarks have published results for both models. Gemini 1.0 Pro has 24 scored results on Noometry and o3-mini has 51.

Related comparisons

Go deeper