Model comparison

Claude Haiku 5.5 vs Gemma 4 31B IT

Claude Haiku 5.5 is the stronger model overall, scoring 49.5 to 43.5 on the Noometry Index.

Last verified October 12, 2026. 5 shared benchmarks.

Claude Haiku 5.5 Anthropic

49.5

Rank #52 Confirmed

Gemma 4 31B IT

Google

43.5

Rank #90 Confirmed

Summary

They share 5 benchmarks with published results for both. Claude Haiku 5.5 scores higher in 5 categories and Gemma 4 31B IT in 0 categories; 5 gaps are clear of the uncertainty.
The widest gap is in math, where Claude Haiku 5.5 leads 73.6 to 43.2.
The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 98.9% for Claude Haiku 5.5 and 73.3% for Gemma 4 31B IT.
Gemma 4 31B IT is cheaper at $0.09 / $0.34 per million input/output tokens, against $0.10 / $0.50 for Claude Haiku 5.5.
Claude Haiku 5.5 accepts more context: 1M tokens versus 262K.
Gemma 4 31B IT has downloadable open weights; the other is API-only.

Side by side

Claude Haiku 5.5 and Gemma 4 31B IT specifications
	Claude Haiku 5.5	Gemma 4 31B IT
Provider	Anthropic	Google
Noometry Index	49.5	43.5
Released	2026-10-07	2026-04-02
Weights	Proprietary	Open
Context window	1M	262K
Max output	128K	33K
Input $ / M tokens	$0.10	$0.09
Output $ / M tokens	$0.50	$0.34
Results tracked	9	35

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude Haiku 5.5 leads

Claude Haiku 5.5: 49.2 (#50), Gemma 4 31B IT: 42.3 (#108)

Coding benchmarks
Benchmark	Claude Haiku 5.5	Gemma 4 31B IT
LMArena WebDev	1587	1366
SciCode	—	43.4%
WeirdML	—	52.3%
LMArena Coding	—	1459
ALE-Bench	—	925.5

Reasoning Claude Haiku 5.5 leads

Claude Haiku 5.5: 35.2 (#69), Gemma 4 31B IT: 27.2 (#122)

Reasoning benchmarks
Benchmark	Claude Haiku 5.5	Gemma 4 31B IT
NYT Connections (extended)	65.7%	70.6%
Kagi LLM Benchmark	—	63.5%
CritPt	—	1.4%
Chess Puzzles	—	5%
Thematic Generalization	—	53%
LMArena Hard Prompts	—	1448
Mystery Game Puzzles	30%	—
DTBench	—	82.7%
LMCA	—	39.3%
Surface Evolver Bench	—	30.6%
Epoch Capabilities Index	—	142.74

Math Claude Haiku 5.5 leads

Claude Haiku 5.5: 73.6 (#18), Gemma 4 31B IT: 43.2 (#81)

Math benchmarks
Benchmark	Claude Haiku 5.5	Gemma 4 31B IT
OTIS Mock AIME 2024-2025	98.9%	73.3%
FrontierMath (Tiers 1-3)	75.1%	—
FrontierMath Tier 4	46.3%	—
LMArena Math	—	1465

Knowledge Claude Haiku 5.5 leads

Claude Haiku 5.5: 50.8 (#70), Gemma 4 31B IT: 37.9 (#151)

Knowledge benchmarks
Benchmark	Claude Haiku 5.5	Gemma 4 31B IT
GPQA Diamond	89.6%	75.8%
SimpleQA Verified	23.8%	10.4%
Vectara Hallucination Rate	—	7.4%
LMArena Expert	—	1465

Multimodal Claude Haiku 5.5 leads

Claude Haiku 5.5: 43.7 (#21), Gemma 4 31B IT: 41.6 (#34)

Multimodal benchmarks
Benchmark	Claude Haiku 5.5	Gemma 4 31B IT
LMArena Vision	—	1277
Furniture Assembly	47.5%	—
LMArena Document	—	1425

Multilingual Not comparable

Claude Haiku 5.5: —, Gemma 4 31B IT: 53.8 (#57)

Multilingual benchmarks
Benchmark	Claude Haiku 5.5	Gemma 4 31B IT
LMArena Non-English	—	1431
LMArena Chinese	—	1476
LMArena French	—	1435
LMArena Russian	—	1460
LMArena Spanish	—	1444

Instruction Following Not comparable

Claude Haiku 5.5: —, Gemma 4 31B IT: 75.5 (#61)

Instruction Following benchmarks
Benchmark	Claude Haiku 5.5	Gemma 4 31B IT
LMArena Instruction Following	—	1433

Long Context Not comparable

Claude Haiku 5.5: —, Gemma 4 31B IT: 44.2 (#71)

Long Context benchmarks
Benchmark	Claude Haiku 5.5	Gemma 4 31B IT
LMArena Longer Query	—	1446

Writing & Preference Not comparable

Claude Haiku 5.5: —, Gemma 4 31B IT: 60.5 (#96)

Writing & Preference benchmarks
Benchmark	Claude Haiku 5.5	Gemma 4 31B IT
LMArena Text	—	1443
LMArena Creative Writing	—	1415
EQ-Bench Creative Writing	—	1368
EQ-Bench 4	—	1120
LMArena Multi-Turn	—	1452

Frequently asked questions

Is Claude Haiku 5.5 better than Gemma 4 31B IT?

Claude Haiku 5.5 is the stronger model overall, scoring 49.5 to 43.5 on the Noometry Index.

Which is cheaper, Claude Haiku 5.5 or Gemma 4 31B IT?

Gemma 4 31B IT is cheaper. It lists at $0.09 per million input tokens and $0.34 per million output tokens; Claude Haiku 5.5 lists at $0.10 and $0.50.

Is Claude Haiku 5.5 or Gemma 4 31B IT better for coding?

Claude Haiku 5.5 scores higher on coding benchmarks: 49.2 versus 42.3 in the Noometry coding category.

Which has the bigger context window?

Claude Haiku 5.5 does, with 1M tokens against 262K.

How many benchmarks do Claude Haiku 5.5 and Gemma 4 31B IT share?

5 benchmarks have published results for both models. Claude Haiku 5.5 has 9 scored results on Noometry and Gemma 4 31B IT has 35.