Noometry

Model comparison

Gemini 1.5 Flash 8B vs o3

o3 is the stronger model overall, scoring 47.5 to 29.9 on the Noometry Index.

Last verified . 21 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.9

Rank #301 Confirmed

o3 OpenAI

47.5

Rank #61 Confirmed

Summary

  • They share 21 benchmarks with published results for both. Gemini 1.5 Flash 8B scores higher in 0 categories and o3 in 9 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where o3 leads 54.6 to 16.0.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 4.6% for Gemini 1.5 Flash 8B and 84.4% for o3.

Side by side

Gemini 1.5 Flash 8Bo3
ProviderGoogleOpenAI
Noometry Index29.947.5
Released2024-10-032025-04-16
WeightsProprietaryProprietary
Context window—200K
Max output—100K
Input $ / M tokens—$2
Output $ / M tokens—$8
Results tracked2163

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3 leads

Gemini 1.5 Flash 8B: 35.5 (#225), o3: 46.8 (#64)

BenchmarkGemini 1.5 Flash 8Bo3
LMArena Coding12181408
SWE-bench Verified—62.3%
SWE-bench Verified (bash only)—58.4%
Aider Polyglot—81.3%
GSO—8.8%
WeirdML—52.4%
CadEval—74%
ALE-Bench—933.55

Agentic & Tool Use Not comparable

Gemini 1.5 Flash 8B: —, o3: 34.5 (#44)

BenchmarkGemini 1.5 Flash 8Bo3
Berkeley Function Calling Leaderboard—63%
GDPval—30.8%
DeepResearch Bench—45.2%
OSWorld—23%
LMArena Search—1144
METR Time Horizons—65.4%

Reasoning o3 leads

Gemini 1.5 Flash 8B: 20.0 (#244), o3: 32.0 (#78)

BenchmarkGemini 1.5 Flash 8Bo3
LMArena Hard Prompts12091402
DTBench50%84.8%
ARC-AGI-2—6.5%
SimpleBench—53.1%
Kagi LLM Benchmark—67.6%
ARC-AGI-1—60.8%
CritPt—1.4%
Chess Puzzles—38%
EnigmaEval—13.1%
Mystery Game Puzzles—29%
LMCA—39.7%
Epoch Capabilities Index—146.86
ForecastBench—62.5

Math o3 leads

Gemini 1.5 Flash 8B: 14.2 (#302), o3: 50.2 (#58)

BenchmarkGemini 1.5 Flash 8Bo3
OTIS Mock AIME 2024-20254.6%84.4%
LMArena Math12071426
FrontierMath (Tiers 1-3)—33.3%
Omni-MATH—71.4%
MATH Level 5—97.8%
FrontierMath (Feb 2025 set)—18.7%
FrontierMath Tier 4 (v1)—2.1%

Knowledge o3 leads

Gemini 1.5 Flash 8B: 16.0 (#289), o3: 54.6 (#52)

BenchmarkGemini 1.5 Flash 8Bo3
GPQA Diamond33%81.8%
LMArena Expert11851402
Humanity's Last Exam—20.3%
SimpleQA Verified—49.4%
MMLU-Pro—85.9%
Confabulations—14.4%
GPQA (HELM)—75.3%

Multimodal o3 leads

Gemini 1.5 Flash 8B: 28.2 (#115), o3: 41.4 (#36)

BenchmarkGemini 1.5 Flash 8Bo3
LMArena Vision10441214
GeoBench—74%
VPCT—52%

Multilingual o3 leads

Gemini 1.5 Flash 8B: 38.5 (#229), o3: 51.7 (#105)

BenchmarkGemini 1.5 Flash 8Bo3
LMArena Non-English12151401
LMArena Chinese12311437
LMArena French12341430
LMArena German12061420
LMArena Japanese11501403
LMArena Korean11401370
LMArena Russian12361406
LMArena Spanish12121395

Instruction Following o3 leads

Gemini 1.5 Flash 8B: 62.8 (#236), o3: 72.8 (#127)

BenchmarkGemini 1.5 Flash 8Bo3
LMArena Instruction Following11991368
IFEval—86.9%

Long Context o3 leads

Gemini 1.5 Flash 8B: 37.0 (#225), o3: 53.3 (#6)

BenchmarkGemini 1.5 Flash 8Bo3
LMArena Longer Query12191372
Fiction.LiveBench—88.9%
CL-bench—17.8%

Writing & Preference o3 leads

Gemini 1.5 Flash 8B: 42.8 (#232), o3: 63.5 (#64)

BenchmarkGemini 1.5 Flash 8Bo3
LMArena Text12261410
LMArena Creative Writing12181359
LMArena Multi-Turn11851405
Short-Story Creative Writing—83.9%
EQ-Bench Creative Writing—1676
WildBench—86.1%

Frequently asked questions

Is Gemini 1.5 Flash 8B better than o3?

o3 is the stronger model overall, scoring 47.5 to 29.9 on the Noometry Index.

Is Gemini 1.5 Flash 8B or o3 better for coding?

o3 scores higher on coding benchmarks: 46.8 versus 35.5 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash 8B and o3 share?

21 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and o3 has 63.

Related comparisons

Go deeper