Noometry

Model comparison

Gemini Advanced 0514 vs o3

o3 is the stronger model overall, scoring 47.5 to 37.7 on the Noometry Index.

Last verified . 17 shared benchmarks.

Gemini Advanced 0514 Google

37.7

Rank #197 Confirmed

o3 OpenAI

47.5

Rank #61 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Gemini Advanced 0514 scores higher in 0 categories and o3 in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where o3 leads 54.6 to 33.3.

Side by side

Gemini Advanced 0514o3
ProviderGoogleOpenAI
Noometry Index37.747.5
Released—2025-04-16
WeightsProprietaryProprietary
Context window—200K
Max output—100K
Input $ / M tokens—$2
Output $ / M tokens—$8
Results tracked1763

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3 leads

Gemini Advanced 0514: 36.6 (#205), o3: 46.8 (#64)

BenchmarkGemini Advanced 0514o3
LMArena Coding12551408
SWE-bench Verified—62.3%
SWE-bench Verified (bash only)—58.4%
Aider Polyglot—81.3%
GSO—8.8%
WeirdML—52.4%
CadEval—74%
ALE-Bench—933.55

Agentic & Tool Use Not comparable

Gemini Advanced 0514: —, o3: 34.5 (#44)

BenchmarkGemini Advanced 0514o3
Berkeley Function Calling Leaderboard—63%
GDPval—30.8%
DeepResearch Bench—45.2%
OSWorld—23%
LMArena Search—1144
METR Time Horizons—65.4%

Reasoning o3 leads

Gemini Advanced 0514: 24.6 (#160), o3: 32.0 (#78)

BenchmarkGemini Advanced 0514o3
LMArena Hard Prompts12481402
ARC-AGI-2—6.5%
SimpleBench—53.1%
Kagi LLM Benchmark—67.6%
ARC-AGI-1—60.8%
CritPt—1.4%
Chess Puzzles—38%
EnigmaEval—13.1%
Mystery Game Puzzles—29%
DTBench—84.8%
LMCA—39.7%
Epoch Capabilities Index—146.86
ForecastBench—62.5

Math o3 leads

Gemini Advanced 0514: 35.5 (#181), o3: 50.2 (#58)

BenchmarkGemini Advanced 0514o3
LMArena Math12721426
FrontierMath (Tiers 1-3)—33.3%
OTIS Mock AIME 2024-2025—84.4%
Omni-MATH—71.4%
MATH Level 5—97.8%
FrontierMath (Feb 2025 set)—18.7%
FrontierMath Tier 4 (v1)—2.1%

Knowledge o3 leads

Gemini Advanced 0514: 33.3 (#205), o3: 54.6 (#52)

BenchmarkGemini Advanced 0514o3
LMArena Expert12151402
GPQA Diamond—81.8%
Humanity's Last Exam—20.3%
SimpleQA Verified—49.4%
MMLU-Pro—85.9%
Confabulations—14.4%
GPQA (HELM)—75.3%

Multimodal Not comparable

Gemini Advanced 0514: —, o3: 41.4 (#36)

BenchmarkGemini Advanced 0514o3
LMArena Vision—1214
GeoBench—74%
VPCT—52%

Multilingual o3 leads

Gemini Advanced 0514: 42.6 (#192), o3: 51.7 (#105)

BenchmarkGemini Advanced 0514o3
LMArena Non-English12741401
LMArena Chinese12741437
LMArena French13071430
LMArena German12751420
LMArena Japanese12421403
LMArena Korean12321370
LMArena Russian12831406
LMArena Spanish12421395

Instruction Following o3 leads

Gemini Advanced 0514: 66.0 (#212), o3: 72.8 (#127)

BenchmarkGemini Advanced 0514o3
LMArena Instruction Following12531368
IFEval—86.9%

Long Context o3 leads

Gemini Advanced 0514: 38.0 (#207), o3: 53.3 (#6)

BenchmarkGemini Advanced 0514o3
LMArena Longer Query12531372
Fiction.LiveBench—88.9%
CL-bench—17.8%

Writing & Preference o3 leads

Gemini Advanced 0514: 48.5 (#200), o3: 63.5 (#64)

BenchmarkGemini Advanced 0514o3
LMArena Text12791410
LMArena Creative Writing12841359
LMArena Multi-Turn12691405
Short-Story Creative Writing—83.9%
EQ-Bench Creative Writing—1676
WildBench—86.1%

Frequently asked questions

Is Gemini Advanced 0514 better than o3?

o3 is the stronger model overall, scoring 47.5 to 37.7 on the Noometry Index.

Is Gemini Advanced 0514 or o3 better for coding?

o3 scores higher on coding benchmarks: 46.8 versus 36.6 in the Noometry coding category.

How many benchmarks do Gemini Advanced 0514 and o3 share?

17 benchmarks have published results for both models. Gemini Advanced 0514 has 17 scored results on Noometry and o3 has 63.

Related comparisons

Go deeper