Noometry

Model comparison

Gemini 2.5 Pro vs Gemma 4 31B IT

Gemini 2.5 Pro is the stronger model overall, scoring 45.0 to 43.5 on the Noometry Index. Gemma 4 31B IT costs 23× less per token, which makes it the better buy when Gemini 2.5 Pro's lead doesn't matter for your workload.

Last verified . 30 shared benchmarks.

Gemini 2.5 Pro Google

45.0

Rank #75 Confirmed

Gemma 4 31B IT Google

43.5

Rank #90 Confirmed

Summary

  • They share 30 benchmarks with published results for both. Gemini 2.5 Pro scores higher in 7 categories and Gemma 4 31B IT in 2 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Gemini 2.5 Pro leads 56.0 to 37.9.
  • The biggest single-benchmark swing is Chess Puzzles: 20% for Gemini 2.5 Pro and 5% for Gemma 4 31B IT.
  • Gemma 4 31B IT is cheaper at $0.09 / $0.34 per million input/output tokens, against $1.25 / $10 for Gemini 2.5 Pro.
  • Gemini 2.5 Pro accepts more context: 1.05M tokens versus 262K.
  • Gemma 4 31B IT has downloadable open weights; the other is API-only.

Side by side

Gemini 2.5 ProGemma 4 31B IT
ProviderGoogleGoogle
Noometry Index45.043.5
Released2025-03-252026-04-02
WeightsProprietaryOpen
Context window1.05M262K
Max output66K33K
Input $ / M tokens$1.25$0.09
Output $ / M tokens$10$0.34
Results tracked7835

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

Gemini 2.5 Pro: 42.4 (#101), Gemma 4 31B IT: 42.3 (#108)

BenchmarkGemini 2.5 ProGemma 4 31B IT
LMArena WebDev12271366
SciCode42.8%43.4%
WeirdML54%52.3%
LMArena Coding14521459
ALE-Bench785.52925.5
SWE-bench Verified57.6%—
SWE-bench Verified (bash only)53.6%—
Aider Polyglot83.1%—
GSO3.9%—
LiveBench Coding85.9%—
CadEval64%—
AlgoTune1.51—

Agentic & Tool Use Not comparable

Gemini 2.5 Pro: 29.2 (#88), Gemma 4 31B IT: —

BenchmarkGemini 2.5 ProGemma 4 31B IT
Terminal-Bench32.6%—
GDPval23.3%—
Remote Labor Index0.8%—
TheAgentCompany30.3%—
τ²-bench Banking13.7%—
DeepResearch Bench42.8%—
BALROG43.3%—
LMArena Search1142—
METR Time Horizons55.4%—
Vending-Bench 2573.64—

Reasoning Gemini 2.5 Pro leads

Gemini 2.5 Pro: 28.8 (#99), Gemma 4 31B IT: 27.2 (#122)

BenchmarkGemini 2.5 ProGemma 4 31B IT
Kagi LLM Benchmark70.3%63.5%
CritPt2%1.4%
Chess Puzzles20%5%
LMArena Hard Prompts14551448
DTBench82.4%82.7%
LMCA34.8%39.3%
Epoch Capabilities Index145.32142.74
ARC-AGI-24.9%—
SimpleBench62.4%—
NYT Connections (extended)—70.6%
ARC-AGI-141%—
EnigmaEval5.6%—
Thematic Generalization—53%
LiveBench Reasoning89.8%—
LiveBench Data Analysis79.9%—
Surface Evolver Bench—30.6%
ForecastBench61.3—
LiveBench82.3%—

Math Gemma 4 31B IT leads

Gemini 2.5 Pro: 32.5 (#213), Gemma 4 31B IT: 43.2 (#81)

BenchmarkGemini 2.5 ProGemma 4 31B IT
OTIS Mock AIME 2024-202584.7%73.3%
LMArena Math14501465
FrontierMath (Tiers 1-3)24.6%—
FrontierMath Tier 40%—
Omni-MATH41.6%—
LiveBench Math90.2%—
MATH Level 595.9%—
FrontierMath (Feb 2025 set)14.1%—
FrontierMath Tier 4 (v1)4.2%—

Knowledge Gemini 2.5 Pro leads

Gemini 2.5 Pro: 56.0 (#46), Gemma 4 31B IT: 37.9 (#151)

BenchmarkGemini 2.5 ProGemma 4 31B IT
GPQA Diamond85.3%75.8%
Vectara Hallucination Rate7%7.4%
LMArena Expert14521465
Humanity's Last Exam21.6%—
SimpleQA Verified—10.4%
MMLU-Pro86.3%—
Confabulations10.6%—
GPQA (HELM)74.9%—

Multimodal Gemini 2.5 Pro leads

Gemini 2.5 Pro: 45.2 (#18), Gemma 4 31B IT: 41.6 (#34)

BenchmarkGemini 2.5 ProGemma 4 31B IT
LMArena Vision12631277
LMArena Document14211425
GeoBench86%—
VPCT48%—
SpatialViz-Bench44.7%—

Multilingual Gemini 2.5 Pro leads

Gemini 2.5 Pro: 55.3 (#31), Gemma 4 31B IT: 53.8 (#57)

BenchmarkGemini 2.5 ProGemma 4 31B IT
LMArena Non-English14511431
LMArena Chinese15071476
LMArena French14721435
LMArena Russian14611460
LMArena Spanish14731444
LMArena German1487—
LMArena Japanese1461—
LMArena Korean1434—

Instruction Following Too close to call

Gemini 2.5 Pro: 75.0 (#75), Gemma 4 31B IT: 75.5 (#61)

BenchmarkGemini 2.5 ProGemma 4 31B IT
LMArena Instruction Following14371433
LiveBench Instruction Following80.6%—
IFEval84%—

Long Context Gemini 2.5 Pro leads

Gemini 2.5 Pro: 59.8 (#5), Gemma 4 31B IT: 44.2 (#71)

BenchmarkGemini 2.5 ProGemma 4 31B IT
LMArena Longer Query14491446
Fiction.LiveBench91.7%—

Writing & Preference Gemini 2.5 Pro leads

Gemini 2.5 Pro: 63.7 (#62), Gemma 4 31B IT: 60.5 (#96)

BenchmarkGemini 2.5 ProGemma 4 31B IT
LMArena Text14581443
LMArena Creative Writing14541415
EQ-Bench Creative Writing14211368
LMArena Multi-Turn14531452
Short-Story Creative Writing83.8%—
WildBench85.7%—
EQ-Bench 4—1120
LiveBench Language67.8%—

Frequently asked questions

Is Gemini 2.5 Pro better than Gemma 4 31B IT?

Gemini 2.5 Pro is the stronger model overall, scoring 45.0 to 43.5 on the Noometry Index. Gemma 4 31B IT costs 23× less per token, which makes it the better buy when Gemini 2.5 Pro's lead doesn't matter for your workload.

Which is cheaper, Gemini 2.5 Pro or Gemma 4 31B IT?

Gemma 4 31B IT is cheaper. It lists at $0.09 per million input tokens and $0.34 per million output tokens; Gemini 2.5 Pro lists at $1.25 and $10.

Is Gemini 2.5 Pro or Gemma 4 31B IT better for coding?

They score almost the same on coding (42.4 vs 42.3); test both on your own repository before choosing.

Which has the bigger context window?

Gemini 2.5 Pro does, with 1.05M tokens against 262K.

How many benchmarks do Gemini 2.5 Pro and Gemma 4 31B IT share?

30 benchmarks have published results for both models. Gemini 2.5 Pro has 78 scored results on Noometry and Gemma 4 31B IT has 35.

Related comparisons

Go deeper