Noometry

Model comparison

Gemini 3 Flash Preview vs o4-mini

Gemini 3 Flash Preview is the stronger model overall, scoring 52.3 to 41.6 on the Noometry Index.

Last verified . 41 shared benchmarks.

Gemini 3 Flash Preview Google

52.3

Rank #40 Confirmed

o4-mini OpenAI

41.6

Rank #132 Confirmed

Summary

  • They share 41 benchmarks with published results for both. Gemini 3 Flash Preview scores higher in 9 categories and o4-mini in 1 category; 9 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Gemini 3 Flash Preview leads 49.2 to 24.6.
  • The biggest single-benchmark swing is SimpleQA Verified: 66.8% for Gemini 3 Flash Preview and 19.6% for o4-mini.
  • Gemini 3 Flash Preview is cheaper at $0.50 / $3 per million input/output tokens, against $1.10 / $4.40 for o4-mini.
  • Gemini 3 Flash Preview accepts more context: 1.05M tokens versus 200K.

Side by side

Gemini 3 Flash Previewo4-mini
ProviderGoogleOpenAI
Noometry Index52.341.6
Released2025-12-172025-04-16
WeightsProprietaryProprietary
Context window1.05M200K
Max output66K100K
Input $ / M tokens$0.50$1.10
Output $ / M tokens$3$4.40
Results tracked5960

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 50.9 (#42), o4-mini: 40.9 (#127)

BenchmarkGemini 3 Flash Previewo4-mini
SWE-bench Verified (bash only)75.8%45%
GSO9.8%3.6%
WeirdML61.6%52.6%
LMArena Coding14601368
ALE-Bench1,367826.17
SWE-bench Verified75.4%—
Aider Polyglot—72%
LMArena WebDev1439—
SWE-bench Multilingual72.7%—
CadEval—62%
AlgoTune—1.72

Agentic & Tool Use Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 38.7 (#29), o4-mini: 32.6 (#61)

BenchmarkGemini 3 Flash Previewo4-mini
Terminal-Bench64.3%—
Berkeley Function Calling Leaderboard—53.2%
GDPval—25.3%
τ²-bench Airline82.5%—
τ²-bench Banking27.3%—
τ²-bench Retail76.8%—
τ²-bench Telecom91.2%—
DeepResearch Bench49.8%—
BALROG48.1%—
GDP.pdf10%—
LMArena Search1198—
METR Time Horizons—63.9%
Vending-Bench 23,635—

Reasoning Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 49.2 (#37), o4-mini: 24.6 (#162)

BenchmarkGemini 3 Flash Previewo4-mini
ARC-AGI-233.6%6.1%
SimpleBench61.1%38.7%
ARC-AGI-184.7%58.7%
Chess Puzzles40%26%
LMArena Hard Prompts14651351
Mystery Game Puzzles26%5%
DTBench89.1%77.6%
LMCA43.1%26.5%
Epoch Capabilities Index151.8145.64
ForecastBench58.561.8
Kagi LLM Benchmark—67.6%
NYT Connections (extended)83.1%—
CritPt—0.6%
EnigmaEval—9.2%

Math Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 51.7 (#55), o4-mini: 40.8 (#89)

BenchmarkGemini 3 Flash Previewo4-mini
FrontierMath (Tiers 1-3)51.2%36.1%
FrontierMath Tier 417.1%4.9%
OTIS Mock AIME 2024-202595.6%81.7%
LMArena Math14731389
FrontierMath (Feb 2025 set)35.6%24.8%
FrontierMath Tier 4 (v1)4.2%6.3%
MathArena Final-Answer Competitions67.6%—
ProofBench15%—
Omni-MATH—72%
MATH Level 5—97.8%

Knowledge Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 58.8 (#33), o4-mini: 43.6 (#91)

BenchmarkGemini 3 Flash Previewo4-mini
GPQA Diamond89.4%79.6%
SimpleQA Verified66.8%19.6%
Vectara Hallucination Rate13.5%18.6%
LMArena Expert14621343
Humanity's Last Exam—18.1%
MMLU-Pro—82%
Confabulations—15.8%
GPQA (HELM)—73.5%

Multimodal Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 45.5 (#16), o4-mini: 40.2 (#49)

BenchmarkGemini 3 Flash Previewo4-mini
LMArena Vision12851194
GeoBench88%64%
VPCT72.6%57.5%
Blueprint-Bench 20%—
LMArena Document1413—

Multilingual Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 55.7 (#27), o4-mini: 47.0 (#154)

BenchmarkGemini 3 Flash Previewo4-mini
LMArena Non-English14581337
LMArena Chinese15111354
LMArena French14771364
LMArena German14971336
LMArena Japanese14891308
LMArena Korean14431312
LMArena Russian14801334
LMArena Spanish14691347

Instruction Following Too close to call

Gemini 3 Flash Preview: 75.7 (#56), o4-mini: 75.2 (#68)

BenchmarkGemini 3 Flash Previewo4-mini
LMArena Instruction Following14371321
IFEval—92.8%

Long Context o4-mini leads

Gemini 3 Flash Preview: 44.4 (#67), o4-mini: 45.5 (#33)

BenchmarkGemini 3 Flash Previewo4-mini
LMArena Longer Query14521315
Fiction.LiveBench—77.8%

Writing & Preference Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 65.5 (#45), o4-mini: 54.0 (#152)

BenchmarkGemini 3 Flash Previewo4-mini
LMArena Text14661353
LMArena Creative Writing14571294
LMArena Multi-Turn14711350
Short-Story Creative Writing—75%
WildBench—85.4%

Frequently asked questions

Is Gemini 3 Flash Preview better than o4-mini?

Gemini 3 Flash Preview is the stronger model overall, scoring 52.3 to 41.6 on the Noometry Index.

Which is cheaper, Gemini 3 Flash Preview or o4-mini?

Gemini 3 Flash Preview is cheaper. It lists at $0.50 per million input tokens and $3 per million output tokens; o4-mini lists at $1.10 and $4.40.

Is Gemini 3 Flash Preview or o4-mini better for coding?

Gemini 3 Flash Preview scores higher on coding benchmarks: 50.9 versus 40.9 in the Noometry coding category.

Which has the bigger context window?

Gemini 3 Flash Preview does, with 1.05M tokens against 200K.

How many benchmarks do Gemini 3 Flash Preview and o4-mini share?

41 benchmarks have published results for both models. Gemini 3 Flash Preview has 59 scored results on Noometry and o4-mini has 60.

Related comparisons

Go deeper