Noometry

Model comparison

Claude 3 Sonnet vs Gemma 4 31B IT

Gemma 4 31B IT is the stronger model overall, scoring 43.2 to 28.8 on the Noometry Index.

Last verified . 20 shared benchmarks.

Claude 3 Sonnet Anthropic

28.8

Rank #311 Confirmed

Gemma 4 31B IT Google

43.2

Rank #85 Confirmed

Summary

  • They share 20 benchmarks with published results for both. Claude 3 Sonnet scores higher in 0 categories and Gemma 4 31B IT in 9 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in math, where Gemma 4 31B IT leads 41.3 to 10.0.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 2.5% for Claude 3 Sonnet and 73.3% for Gemma 4 31B IT.
  • Gemma 4 31B IT has downloadable open weights; the other is API-only.

Side by side

Claude 3 SonnetGemma 4 31B IT
ProviderAnthropicGoogle
Noometry Index28.843.2
Released2024-02-292026-04-02
WeightsProprietaryOpen
Context window—262K
Max output—33K
Input $ / M tokens—$0.09
Output $ / M tokens—$0.34
Results tracked3035

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemma 4 31B IT leads

Claude 3 Sonnet: 29.2 (#296), Gemma 4 31B IT: 41.9 (#98)

BenchmarkClaude 3 SonnetGemma 4 31B IT
WeirdML10.2%52.3%
LMArena Coding12231459
LMArena WebDev—1364
SciCode—43.4%
BigCodeBench Instruct42.7%—
BigCodeBench Complete53.8%—
ALE-Bench—925.5
HumanEval+64%—
MBPP+69.3%—

Reasoning Gemma 4 31B IT leads

Claude 3 Sonnet: 19.7 (#232), Gemma 4 31B IT: 26.4 (#119)

BenchmarkClaude 3 SonnetGemma 4 31B IT
LMArena Hard Prompts11971448
DTBench53.6%82.7%
Epoch Capabilities Index120.69142.74
Kagi LLM Benchmark—63.5%
NYT Connections (extended)—70.6%
CritPt—1.4%
Chess Puzzles—5%
Thematic Generalization—53%
LMCA—39.3%
Surface Evolver Bench—30.6%
WinoGrande75.1%—

Math Gemma 4 31B IT leads

Claude 3 Sonnet: 10.0 (#305), Gemma 4 31B IT: 41.3 (#72)

BenchmarkClaude 3 SonnetGemma 4 31B IT
OTIS Mock AIME 2024-20252.5%73.3%
LMArena Math12131464
MATH Level 518.2%—

Knowledge Gemma 4 31B IT leads

Claude 3 Sonnet: 21.1 (#271), Gemma 4 31B IT: 37.9 (#141)

BenchmarkClaude 3 SonnetGemma 4 31B IT
GPQA Diamond40.6%75.8%
LMArena Expert11731465
SimpleQA Verified—10.4%
Vectara Hallucination Rate—7.4%
MMLU75.9%—

Multimodal Gemma 4 31B IT leads

Claude 3 Sonnet: 24.7 (#120), Gemma 4 31B IT: 41.0 (#32)

BenchmarkClaude 3 SonnetGemma 4 31B IT
LMArena Vision9841278
LMArena Document—1425

Multilingual Gemma 4 31B IT leads

Claude 3 Sonnet: 38.0 (#229), Gemma 4 31B IT: 54.0 (#52)

BenchmarkClaude 3 SonnetGemma 4 31B IT
LMArena Non-English12051431
LMArena Chinese11881478
LMArena French12281436
LMArena Russian12271462
LMArena Spanish12041443
LMArena German1204—
LMArena Japanese1131—
LMArena Korean1128—

Instruction Following Gemma 4 31B IT leads

Claude 3 Sonnet: 63.1 (#230), Gemma 4 31B IT: 75.6 (#58)

BenchmarkClaude 3 SonnetGemma 4 31B IT
LMArena Instruction Following12001433

Long Context Gemma 4 31B IT leads

Claude 3 Sonnet: 36.7 (#223), Gemma 4 31B IT: 44.2 (#67)

BenchmarkClaude 3 SonnetGemma 4 31B IT
LMArena Longer Query12111446

Writing & Preference Gemma 4 31B IT leads

Claude 3 Sonnet: 42.3 (#233), Gemma 4 31B IT: 60.7 (#92)

BenchmarkClaude 3 SonnetGemma 4 31B IT
LMArena Text12181443
LMArena Creative Writing11861416
LMArena Multi-Turn12271452
EQ-Bench Creative Writing—1368
EQ-Bench 4—1120

Frequently asked questions

Is Claude 3 Sonnet better than Gemma 4 31B IT?

Gemma 4 31B IT is the stronger model overall, scoring 43.2 to 28.8 on the Noometry Index.

Is Claude 3 Sonnet or Gemma 4 31B IT better for coding?

Gemma 4 31B IT scores higher on coding benchmarks: 41.9 versus 29.2 in the Noometry coding category.

How many benchmarks do Claude 3 Sonnet and Gemma 4 31B IT share?

20 benchmarks have published results for both models. Claude 3 Sonnet has 30 scored results on Noometry and Gemma 4 31B IT has 35.

Related comparisons

Go deeper