Noometry

Model comparison

Gemini 3 Flash Preview vs Phi-4

Gemini 3 Flash Preview is the stronger model overall, scoring 52.3 to 30.9 on the Noometry Index. Phi-4 costs 13× less per token, which makes it the better buy when Gemini 3 Flash Preview's lead doesn't matter for your workload.

Last verified . 23 shared benchmarks.

Gemini 3 Flash Preview Google

52.3

Rank #33 Confirmed

Phi-4 Microsoft

30.9

Rank #269 Confirmed

Summary

  • They share 23 benchmarks with published results for both. Gemini 3 Flash Preview scores higher in 9 categories and Phi-4 in 0 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Gemini 3 Flash Preview leads 49.2 to 17.0.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 95.6% for Gemini 3 Flash Preview and 13.8% for Phi-4.
  • Phi-4 is cheaper at $0.07 / $0.14 per million input/output tokens, against $0.50 / $3 for Gemini 3 Flash Preview.
  • Gemini 3 Flash Preview accepts more context: 1.05M tokens versus 128K.
  • Phi-4 has downloadable open weights; the other is API-only.

Side by side

Gemini 3 Flash PreviewPhi-4
ProviderGoogleMicrosoft
Noometry Index52.330.9
Released2025-12-172024-12-11
WeightsProprietaryOpen
Context window1.05M128K
Max output66K4K
Input $ / M tokens$0.50$0.07
Output $ / M tokens$3$0.14
Results tracked5937

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 51.5 (#35), Phi-4: 33.8 (#237)

BenchmarkGemini 3 Flash PreviewPhi-4
LMArena Coding14601231
SWE-bench Verified75.4%—
SWE-bench Verified (bash only)75.8%—
LMArena WebDev1439—
SWE-bench Multilingual72.7%—
GSO9.8%—
WeirdML61.6%—
BigCodeBench Instruct—45.5%
LiveBench Coding—30.7%
BigCodeBench Complete—55.4%
ALE-Bench1,367—

Agentic & Tool Use Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 38.6 (#27), Phi-4: 22.6 (#120)

BenchmarkGemini 3 Flash PreviewPhi-4
BALROG48.1%11.6%
Terminal-Bench64.3%—
Berkeley Function Calling Leaderboard—28.8%
τ²-bench Airline82.5%—
τ²-bench Banking27.3%—
τ²-bench Retail76.8%—
τ²-bench Telecom91.2%—
DeepResearch Bench49.8%—
GDP.pdf10%—
LMArena Search1198—
Vending-Bench 23,635—

Reasoning Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 49.2 (#35), Phi-4: 17.0 (#287)

BenchmarkGemini 3 Flash PreviewPhi-4
Chess Puzzles40%1%
LMArena Hard Prompts14651220
Epoch Capabilities Index151.83130.43
ARC-AGI-233.6%—
SimpleBench61.1%—
NYT Connections (extended)83.1%—
ARC-AGI-184.7%—
LiveBench Reasoning—47.8%
Mystery Game Puzzles26%—
DTBench89.1%—
LiveBench Data Analysis—45.2%
LMCA43.1%—
ForecastBench58.5—
LiveBench—41.6%

Math Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 51.4 (#42), Phi-4: 19.6 (#279)

BenchmarkGemini 3 Flash PreviewPhi-4
OTIS Mock AIME 2024-202595.6%13.8%
LMArena Math14721246
FrontierMath (Tiers 1-3)51.2%—
FrontierMath Tier 417.1%—
MathArena Final-Answer Competitions67.6%—
ProofBench15%—
LiveBench Math—42%
MATH Level 5—64.9%
FrontierMath (Feb 2025 set)35.6%—
FrontierMath Tier 4 (v1)4.2%—

Knowledge Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 58.8 (#30), Phi-4: 32.5 (#204)

BenchmarkGemini 3 Flash PreviewPhi-4
GPQA Diamond89.4%56.1%
Vectara Hallucination Rate13.5%3.7%
LMArena Expert14611203
SimpleQA Verified66.8%—
Confabulations—29.4%
MMLU—84.8%

Multimodal Not comparable

Gemini 3 Flash Preview: 45.3 (#15), Phi-4: —

BenchmarkGemini 3 Flash PreviewPhi-4
LMArena Vision1285—
GeoBench88%—
VPCT72.6%—
Blueprint-Bench 20%—
LMArena Document1413—

Multilingual Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 55.9 (#26), Phi-4: 37.4 (#232)

BenchmarkGemini 3 Flash PreviewPhi-4
LMArena Non-English14581197
LMArena Chinese15101211
LMArena French14781223
LMArena German14971222
LMArena Japanese14881158
LMArena Korean14431151
LMArena Russian14801209
LMArena Spanish14701234

Instruction Following Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 75.8 (#52), Phi-4: 60.6 (#246)

BenchmarkGemini 3 Flash PreviewPhi-4
LMArena Instruction Following14371201
LiveBench Instruction Following—58.4%

Long Context Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 44.4 (#63), Phi-4: 36.9 (#221)

BenchmarkGemini 3 Flash PreviewPhi-4
LMArena Longer Query14521217

Writing & Preference Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 65.6 (#44), Phi-4: 40.7 (#239)

BenchmarkGemini 3 Flash PreviewPhi-4
LMArena Text14661217
LMArena Creative Writing14571182
LMArena Multi-Turn14711206
Short-Story Creative Writing—62.6%
LiveBench Language—25.6%

Frequently asked questions

Is Gemini 3 Flash Preview better than Phi-4?

Gemini 3 Flash Preview is the stronger model overall, scoring 52.3 to 30.9 on the Noometry Index. Phi-4 costs 13× less per token, which makes it the better buy when Gemini 3 Flash Preview's lead doesn't matter for your workload.

Which is cheaper, Gemini 3 Flash Preview or Phi-4?

Phi-4 is cheaper. It lists at $0.07 per million input tokens and $0.14 per million output tokens; Gemini 3 Flash Preview lists at $0.50 and $3.

Is Gemini 3 Flash Preview or Phi-4 better for coding?

Gemini 3 Flash Preview scores higher on coding benchmarks: 51.5 versus 33.8 in the Noometry coding category.

Which has the bigger context window?

Gemini 3 Flash Preview does, with 1.05M tokens against 128K.

How many benchmarks do Gemini 3 Flash Preview and Phi-4 share?

23 benchmarks have published results for both models. Gemini 3 Flash Preview has 59 scored results on Noometry and Phi-4 has 37.

Related comparisons

Go deeper