Noometry

Model comparison

Gemma 4 26B A4B IT vs Phi 3 Mini 128k Instruct

Gemma 4 26B A4B IT is the stronger model overall, scoring 43.5 to 29.7 on the Noometry Index.

Last verified . 14 shared benchmarks.

Gemma 4 26B A4B IT Google

43.5

Rank #92 Confirmed

Phi 3 Mini 128k Instruct Microsoft

29.7

Rank #305 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Gemma 4 26B A4B IT scores higher in 8 categories and Phi 3 Mini 128k Instruct in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Gemma 4 26B A4B IT leads 58.6 to 27.1.

Side by side

Gemma 4 26B A4B ITPhi 3 Mini 128k Instruct
ProviderGoogleMicrosoft
Noometry Index43.529.7
Released2026-04-022024-04-23
WeightsOpenOpen
Context window262K—
Max output33K—
Input $ / M tokens$0.0675—
Output $ / M tokens$0.23—
Results tracked2819

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 39.0 (#164), Phi 3 Mini 128k Instruct: 28.8 (#312)

BenchmarkGemma 4 26B A4B ITPhi 3 Mini 128k Instruct
LMArena Coding14471039
LMArena WebDev1359—
SciCode40%—
WeirdML35.2%—
BigCodeBench Instruct—29.6%
BigCodeBench Complete—40.6%
ALE-Bench927.17—

Reasoning Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 21.8 (#213), Phi 3 Mini 128k Instruct: 19.6 (#256)

BenchmarkGemma 4 26B A4B ITPhi 3 Mini 128k Instruct
LMArena Hard Prompts14391028
CritPt0%—
Chess Puzzles6%—
DTBench74.9%—
LMCA29.7%—
Epoch Capabilities Index141.85—

Math Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 47.6 (#67), Phi 3 Mini 128k Instruct: 31.6 (#222)

BenchmarkGemma 4 26B A4B ITPhi 3 Mini 128k Instruct
LMArena Math14701089
OTIS Mock AIME 2024-202582.2%—

Knowledge Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 45.7 (#85), Phi 3 Mini 128k Instruct: 26.8 (#254)

BenchmarkGemma 4 26B A4B ITPhi 3 Mini 128k Instruct
LMArena Expert1447984
GPQA Diamond73.2%—
Vectara Hallucination Rate5.2%—

Multimodal Not comparable

Gemma 4 26B A4B IT: 40.6 (#46), Phi 3 Mini 128k Instruct: —

BenchmarkGemma 4 26B A4B ITPhi 3 Mini 128k Instruct
LMArena Vision1260—

Multilingual Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 53.1 (#71), Phi 3 Mini 128k Instruct: 25.2 (#285)

BenchmarkGemma 4 26B A4B ITPhi 3 Mini 128k Instruct
LMArena Non-English14211000
LMArena Chinese14951016
LMArena French14601039
LMArena Russian14341004
LMArena Spanish14171059
LMArena German—1006
LMArena Japanese—899
LMArena Korean—856

Instruction Following Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 74.8 (#82), Phi 3 Mini 128k Instruct: 51.8 (#294)

BenchmarkGemma 4 26B A4B ITPhi 3 Mini 128k Instruct
LMArena Instruction Following14201023

Long Context Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 43.6 (#91), Phi 3 Mini 128k Instruct: 30.4 (#289)

BenchmarkGemma 4 26B A4B ITPhi 3 Mini 128k Instruct
LMArena Longer Query1428996

Writing & Preference Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 58.6 (#115), Phi 3 Mini 128k Instruct: 27.1 (#301)

BenchmarkGemma 4 26B A4B ITPhi 3 Mini 128k Instruct
LMArena Text14341050
LMArena Creative Writing14021024
LMArena Multi-Turn1441989
EQ-Bench Creative Writing1305—

Frequently asked questions

Is Gemma 4 26B A4B IT better than Phi 3 Mini 128k Instruct?

Gemma 4 26B A4B IT is the stronger model overall, scoring 43.5 to 29.7 on the Noometry Index.

Is Gemma 4 26B A4B IT or Phi 3 Mini 128k Instruct better for coding?

Gemma 4 26B A4B IT scores higher on coding benchmarks: 39.0 versus 28.8 in the Noometry coding category.

How many benchmarks do Gemma 4 26B A4B IT and Phi 3 Mini 128k Instruct share?

14 benchmarks have published results for both models. Gemma 4 26B A4B IT has 28 scored results on Noometry and Phi 3 Mini 128k Instruct has 19.

Related comparisons

Go deeper