Noometry

Model comparison

Gemini 1.5 Flash 8B vs o3-mini

o3-mini is the stronger model overall, scoring 36.7 to 29.9 on the Noometry Index.

Last verified . 20 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.9

Rank #301 Confirmed

o3-mini OpenAI

36.7

Rank #212 Confirmed

Summary

  • They share 20 benchmarks with published results for both. Gemini 1.5 Flash 8B scores higher in 2 categories and o3-mini in 6 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where o3-mini leads 38.3 to 16.0.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 4.6% for Gemini 1.5 Flash 8B and 76.9% for o3-mini.

Side by side

Gemini 1.5 Flash 8Bo3-mini
ProviderGoogleOpenAI
Noometry Index29.936.7
Released2024-10-032024-12-20
WeightsProprietaryProprietary
Context window—200K
Max output—100K
Input $ / M tokens—$1.10
Output $ / M tokens—$4.40
Results tracked2151

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3-mini leads

Gemini 1.5 Flash 8B: 35.5 (#225), o3-mini: 40.8 (#132)

BenchmarkGemini 1.5 Flash 8Bo3-mini
LMArena Coding12181378
Aider Polyglot—60.4%
SciCode—39.8%
GSO—1.3%
WeirdML—43.7%
LiveBench Coding—82.7%
CadEval—54%

Agentic & Tool Use Not comparable

Gemini 1.5 Flash 8B: —, o3-mini: 29.6 (#84)

BenchmarkGemini 1.5 Flash 8Bo3-mini
Cybench—22.5%

Reasoning Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 20.0 (#244), o3-mini: 16.3 (#305)

BenchmarkGemini 1.5 Flash 8Bo3-mini
LMArena Hard Prompts12091366
DTBench50%68.8%
ARC-AGI-2—3%
SimpleBench—22.8%
ARC-AGI-1—34.5%
CritPt—0.3%
Chess Puzzles—17%
LiveBench Reasoning—89.6%
Mystery Game Puzzles—7%
LiveBench Data Analysis—70.6%
LMCA—19%
Epoch Capabilities Index—140.34
ForecastBench—59.6
LiveBench—75.9%

Math o3-mini leads

Gemini 1.5 Flash 8B: 14.2 (#302), o3-mini: 28.1 (#244)

BenchmarkGemini 1.5 Flash 8Bo3-mini
OTIS Mock AIME 2024-20254.6%76.9%
LMArena Math12071396
FrontierMath (Tiers 1-3)—18.6%
FrontierMath Tier 4—0%
LiveBench Math—77.3%
MATH Level 5—96.5%
FrontierMath (Feb 2025 set)—12.4%
FrontierMath Tier 4 (v1)—4.2%

Knowledge o3-mini leads

Gemini 1.5 Flash 8B: 16.0 (#289), o3-mini: 38.3 (#146)

BenchmarkGemini 1.5 Flash 8Bo3-mini
GPQA Diamond33%77%
LMArena Expert11851364
SimpleQA Verified—15.3%
Confabulations—17.9%

Multimodal Not comparable

Gemini 1.5 Flash 8B: 28.2 (#115), o3-mini: —

BenchmarkGemini 1.5 Flash 8Bo3-mini
LMArena Vision1044—

Multilingual o3-mini leads

Gemini 1.5 Flash 8B: 38.5 (#229), o3-mini: 45.7 (#164)

BenchmarkGemini 1.5 Flash 8Bo3-mini
LMArena Non-English12151319
LMArena Chinese12311379
LMArena French12341334
LMArena German12061303
LMArena Japanese11501286
LMArena Korean11401314
LMArena Russian12361304
LMArena Spanish12121321

Instruction Following o3-mini leads

Gemini 1.5 Flash 8B: 62.8 (#236), o3-mini: 75.1 (#72)

BenchmarkGemini 1.5 Flash 8Bo3-mini
LMArena Instruction Following11991337
LiveBench Instruction Following—84.4%

Long Context Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 37.0 (#225), o3-mini: 33.8 (#256)

BenchmarkGemini 1.5 Flash 8Bo3-mini
LMArena Longer Query12191343
Fiction.LiveBench—50%

Writing & Preference o3-mini leads

Gemini 1.5 Flash 8B: 42.8 (#232), o3-mini: 50.3 (#182)

BenchmarkGemini 1.5 Flash 8Bo3-mini
LMArena Text12261337
LMArena Creative Writing12181286
LMArena Multi-Turn11851320
Short-Story Creative Writing—61.7%
LiveBench Language—50.7%

Frequently asked questions

Is Gemini 1.5 Flash 8B better than o3-mini?

o3-mini is the stronger model overall, scoring 36.7 to 29.9 on the Noometry Index.

Is Gemini 1.5 Flash 8B or o3-mini better for coding?

o3-mini scores higher on coding benchmarks: 40.8 versus 35.5 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash 8B and o3-mini share?

20 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and o3-mini has 51.

Related comparisons

Go deeper