Noometry

Model comparison

Gemma 4 31B IT vs Step 2 16k Exp 202412

Gemma 4 31B IT is the stronger model overall, scoring 43.5 to 39.2 on the Noometry Index.

Last verified . 12 shared benchmarks.

Gemma 4 31B IT Google

43.5

Rank #90 Confirmed

Step 2 16k Exp 202412 StepFun

39.2

Rank #171 Confirmed

Summary

  • They share 12 benchmarks with published results for both. Gemma 4 31B IT scores higher in 8 categories and Step 2 16k Exp 202412 in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in multilingual, where Gemma 4 31B IT leads 53.8 to 43.7.
  • Gemma 4 31B IT has downloadable open weights; the other is API-only.

Side by side

Gemma 4 31B ITStep 2 16k Exp 202412
ProviderGoogleStepFun
Noometry Index43.539.2
Released2026-04-02—
WeightsOpenProprietary
Context window262K—
Max output33K—
Input $ / M tokens$0.09—
Output $ / M tokens$0.34—
Results tracked3512

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemma 4 31B IT leads

Gemma 4 31B IT: 42.3 (#108), Step 2 16k Exp 202412: 38.5 (#174)

BenchmarkGemma 4 31B ITStep 2 16k Exp 202412
LMArena Coding14591316
LMArena WebDev1366—
SciCode43.4%—
WeirdML52.3%—
ALE-Bench925.5—

Reasoning Gemma 4 31B IT leads

Gemma 4 31B IT: 27.2 (#122), Step 2 16k Exp 202412: 25.9 (#142)

BenchmarkGemma 4 31B ITStep 2 16k Exp 202412
LMArena Hard Prompts14481299
Kagi LLM Benchmark63.5%—
NYT Connections (extended)70.6%—
CritPt1.4%—
Chess Puzzles5%—
Thematic Generalization53%—
DTBench82.7%—
LMCA39.3%—
Surface Evolver Bench30.6%—
Epoch Capabilities Index142.74—

Math Gemma 4 31B IT leads

Gemma 4 31B IT: 43.2 (#81), Step 2 16k Exp 202412: 36.3 (#169)

BenchmarkGemma 4 31B ITStep 2 16k Exp 202412
LMArena Math14651304
OTIS Mock AIME 2024-202573.3%—

Knowledge Gemma 4 31B IT leads

Gemma 4 31B IT: 37.9 (#151), Step 2 16k Exp 202412: 35.2 (#187)

BenchmarkGemma 4 31B ITStep 2 16k Exp 202412
LMArena Expert14651279
GPQA Diamond75.8%—
SimpleQA Verified10.4%—
Vectara Hallucination Rate7.4%—

Multimodal Not comparable

Gemma 4 31B IT: 41.6 (#34), Step 2 16k Exp 202412: —

BenchmarkGemma 4 31B ITStep 2 16k Exp 202412
LMArena Vision1277—
LMArena Document1425—

Multilingual Gemma 4 31B IT leads

Gemma 4 31B IT: 53.8 (#57), Step 2 16k Exp 202412: 43.7 (#181)

BenchmarkGemma 4 31B ITStep 2 16k Exp 202412
LMArena Non-English14311290
LMArena Chinese14761331
LMArena Russian14601324
LMArena French1435—
LMArena Spanish1444—

Instruction Following Gemma 4 31B IT leads

Gemma 4 31B IT: 75.5 (#61), Step 2 16k Exp 202412: 67.9 (#192)

BenchmarkGemma 4 31B ITStep 2 16k Exp 202412
LMArena Instruction Following14331287

Long Context Gemma 4 31B IT leads

Gemma 4 31B IT: 44.2 (#71), Step 2 16k Exp 202412: 39.7 (#170)

BenchmarkGemma 4 31B ITStep 2 16k Exp 202412
LMArena Longer Query14461307

Writing & Preference Gemma 4 31B IT leads

Gemma 4 31B IT: 60.5 (#96), Step 2 16k Exp 202412: 52.3 (#173)

BenchmarkGemma 4 31B ITStep 2 16k Exp 202412
LMArena Text14431321
LMArena Creative Writing14151328
LMArena Multi-Turn14521292
EQ-Bench Creative Writing1368—
EQ-Bench 41120—

Frequently asked questions

Is Gemma 4 31B IT better than Step 2 16k Exp 202412?

Gemma 4 31B IT is the stronger model overall, scoring 43.5 to 39.2 on the Noometry Index.

Is Gemma 4 31B IT or Step 2 16k Exp 202412 better for coding?

Gemma 4 31B IT scores higher on coding benchmarks: 42.3 versus 38.5 in the Noometry coding category.

How many benchmarks do Gemma 4 31B IT and Step 2 16k Exp 202412 share?

12 benchmarks have published results for both models. Gemma 4 31B IT has 35 scored results on Noometry and Step 2 16k Exp 202412 has 12.

Related comparisons

Go deeper