Model comparison

Claude Haiku 5.5 vs Gemma 3 4B

Claude Haiku 5.5 is the stronger model overall, scoring 49.5 to 28.1 on the Noometry Index. Gemma 3 4B costs 4.0× less per token, which makes it the better buy when Claude Haiku 5.5's lead doesn't matter for your workload.

Last verified October 12, 2026. 2 shared benchmarks.

Claude Haiku 5.5 Anthropic

49.5

Rank #52 Confirmed

Gemma 3 4B

Google

28.1

Rank #326 Confirmed

Summary

They share 2 benchmarks with published results for both. Claude Haiku 5.5 scores higher in 4 categories and Gemma 3 4B in 0 categories; 4 gaps are clear of the uncertainty.
The widest gap is in math, where Claude Haiku 5.5 leads 73.6 to 16.8.
The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 98.9% for Claude Haiku 5.5 and 7.5% for Gemma 3 4B.
Gemma 3 4B is cheaper at $0.04 / $0.08 per million input/output tokens, against $0.10 / $0.50 for Claude Haiku 5.5.
Claude Haiku 5.5 accepts more context: 1M tokens versus 131K.
Gemma 3 4B has downloadable open weights; the other is API-only.

Side by side

Claude Haiku 5.5 and Gemma 3 4B specifications
	Claude Haiku 5.5	Gemma 3 4B
Provider	Anthropic	Google
Noometry Index	49.5	28.1
Released	2026-10-07	2025-03-12
Weights	Proprietary	Open
Context window	1M	131K
Max output	128K	4K
Input $ / M tokens	$0.10	$0.04
Output $ / M tokens	$0.50	$0.08
Results tracked	9	22

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude Haiku 5.5 leads

Claude Haiku 5.5: 49.2 (#50), Gemma 3 4B: 35.9 (#215)

Coding benchmarks
Benchmark	Claude Haiku 5.5	Gemma 3 4B
LMArena WebDev	1587	—
LMArena Coding	—	1230

Agentic & Tool Use Not comparable

Claude Haiku 5.5: —, Gemma 3 4B: 20.9 (#142)

Agentic & Tool Use benchmarks
Benchmark	Claude Haiku 5.5	Gemma 3 4B
Berkeley Function Calling Leaderboard	—	19.6%

Reasoning Claude Haiku 5.5 leads

Claude Haiku 5.5: 35.2 (#69), Gemma 3 4B: 13.2 (#335)

Reasoning benchmarks
Benchmark	Claude Haiku 5.5	Gemma 3 4B
Kagi LLM Benchmark	—	25.2%
NYT Connections (extended)	65.7%	—
Chess Puzzles	—	0%
LMArena Hard Prompts	—	1253
Mystery Game Puzzles	30%	—
DTBench	—	50.9%
LMCA	—	2.8%
Epoch Capabilities Index	—	116.02

Math Claude Haiku 5.5 leads

Claude Haiku 5.5: 73.6 (#18), Gemma 3 4B: 16.8 (#292)

Math benchmarks
Benchmark	Claude Haiku 5.5	Gemma 3 4B
OTIS Mock AIME 2024-2025	98.9%	7.5%
FrontierMath (Tiers 1-3)	75.1%	—
FrontierMath Tier 4	46.3%	—
LMArena Math	—	1239

Knowledge Claude Haiku 5.5 leads

Claude Haiku 5.5: 50.8 (#70), Gemma 3 4B: 11.8 (#299)

Knowledge benchmarks
Benchmark	Claude Haiku 5.5	Gemma 3 4B
GPQA Diamond	89.6%	23.2%
SimpleQA Verified	23.8%	—
Vectara Hallucination Rate	—	6.4%
LMArena Expert	—	1223

Multimodal Not comparable

Claude Haiku 5.5: 43.7 (#21), Gemma 3 4B: —

Multimodal benchmarks
Benchmark	Claude Haiku 5.5	Gemma 3 4B
Furniture Assembly	47.5%	—

Multilingual Not comparable

Claude Haiku 5.5: —, Gemma 3 4B: 42.5 (#194)

Multilingual benchmarks
Benchmark	Claude Haiku 5.5	Gemma 3 4B
LMArena Non-English	—	1273
LMArena German	—	1281
LMArena Russian	—	1294

Instruction Following Not comparable

Claude Haiku 5.5: —, Gemma 3 4B: 65.2 (#225)

Instruction Following benchmarks
Benchmark	Claude Haiku 5.5	Gemma 3 4B
LMArena Instruction Following	—	1239

Long Context Not comparable

Claude Haiku 5.5: —, Gemma 3 4B: 38.7 (#194)

Long Context benchmarks
Benchmark	Claude Haiku 5.5	Gemma 3 4B
LMArena Longer Query	—	1273

Writing & Preference Not comparable

Claude Haiku 5.5: —, Gemma 3 4B: 42.0 (#239)

Writing & Preference benchmarks
Benchmark	Claude Haiku 5.5	Gemma 3 4B
LMArena Text	—	1291
LMArena Creative Writing	—	1271
EQ-Bench Creative Writing	—	1068
LMArena Multi-Turn	—	1255

Frequently asked questions

Is Claude Haiku 5.5 better than Gemma 3 4B?

Which is cheaper, Claude Haiku 5.5 or Gemma 3 4B?

Gemma 3 4B is cheaper. It lists at $0.04 per million input tokens and $0.08 per million output tokens; Claude Haiku 5.5 lists at $0.10 and $0.50.

Is Claude Haiku 5.5 or Gemma 3 4B better for coding?

Claude Haiku 5.5 scores higher on coding benchmarks: 49.2 versus 35.9 in the Noometry coding category.

Which has the bigger context window?

Claude Haiku 5.5 does, with 1M tokens against 131K.

How many benchmarks do Claude Haiku 5.5 and Gemma 3 4B share?

2 benchmarks have published results for both models. Claude Haiku 5.5 has 9 scored results on Noometry and Gemma 3 4B has 22.