Model comparison

Gemma 3 1B vs GPT-5.1

GPT-5.1 is the stronger model overall, scoring 49.0 to 21.1 on the Noometry Index.

Last verified October 12, 2026. 3 shared benchmarks.

Gemma 3 1B

Google

21.1

Rank #353 Reported

GPT-5.1 OpenAI

49.0

Rank #53 Confirmed

Summary

They share 3 benchmarks with published results for both. Gemma 3 1B scores higher in 0 categories and GPT-5.1 in 4 categories; 4 gaps are clear of the uncertainty.
The widest gap is in knowledge, where GPT-5.1 leads 50.6 to 7.0.
The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 1.1% for Gemma 3 1B and 88.6% for GPT-5.1.
Gemma 3 1B has downloadable open weights; the other is API-only.

Side by side

Gemma 3 1B and GPT-5.1 specifications
	Gemma 3 1B	GPT-5.1
Provider	Google	OpenAI
Noometry Index	21.1	49.0
Released	2025-03-12	2025-11-13
Weights	Open	Proprietary
Context window	—	400K
Max output	—	128K
Input $ / M tokens	—	$1.25
Output $ / M tokens	—	$10
Results tracked	4	63

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Gemma 3 1B: —, GPT-5.1: 46.4 (#66)

Coding benchmarks
Benchmark	Gemma 3 1B	GPT-5.1
SWE-bench Verified	—	68%
SWE-bench Verified (bash only)	—	66%
LMArena WebDev	—	1395
SciCode	—	43.3%
GSO	—	13.7%
WeirdML	—	60.8%
LiveBench Coding	—	72.5%
LMArena Coding	—	1454
ALE-Bench	—	1,192

Agentic & Tool Use GPT-5.1 leads

Gemma 3 1B: 13.7 (#152), GPT-5.1: 32.7 (#60)

Agentic & Tool Use benchmarks
Benchmark	Gemma 3 1B	GPT-5.1
Terminal-Bench	—	47.6%
Berkeley Function Calling Leaderboard	7.2%	—
DeepResearch Bench	—	42.8%
LMArena Search	—	1199
Vending-Bench 2	—	1,473

Reasoning GPT-5.1 leads

Gemma 3 1B: 19.2 (#264), GPT-5.1: 39.8 (#58)

Reasoning benchmarks
Benchmark	Gemma 3 1B	GPT-5.1
Chess Puzzles	0%	32%
ARC-AGI-2	—	17.6%
SimpleBench	—	53.2%
ARC-AGI-1	—	72.8%
CritPt	—	4.9%
EnigmaEval	—	11.2%
LiveBench Reasoning	—	95.8%
LMArena Hard Prompts	—	1457
Mystery Game Puzzles	—	19%
DTBench	—	90.1%
LiveBench Data Analysis	—	72.1%
LMCA	—	43.9%
Epoch Capabilities Index	—	149.64
ForecastBench	—	58.1
LiveBench	—	78.8%

Math GPT-5.1 leads

Gemma 3 1B: 10.2 (#316), GPT-5.1: 52.2 (#51)

Math benchmarks
Benchmark	Gemma 3 1B	GPT-5.1
OTIS Mock AIME 2024-2025	1.1%	88.6%
Omni-MATH	—	46.4%
LiveBench Math	—	94.5%
LMArena Math	—	1447
FrontierMath (Feb 2025 set)	—	31%
FrontierMath Tier 4 (v1)	—	12.5%

Knowledge GPT-5.1 leads

Gemma 3 1B: 7.0 (#314), GPT-5.1: 50.6 (#71)

Knowledge benchmarks
Benchmark	Gemma 3 1B	GPT-5.1
GPQA Diamond	19.9%	87.6%
Humanity's Last Exam	—	23.7%
SimpleQA Verified	—	48%
MMLU-Pro	—	57.9%
Vectara Hallucination Rate	—	10.9%
GPQA (HELM)	—	44.2%
LMArena Expert	—	1470

Multimodal Not comparable

Gemma 3 1B: —, GPT-5.1: 44.8 (#19)

Multimodal benchmarks
Benchmark	Gemma 3 1B	GPT-5.1
LMArena Vision	—	1250
VPCT	—	58.7%
LMArena Document	—	1403

Multilingual Not comparable

Gemma 3 1B: —, GPT-5.1: 53.8 (#56)

Multilingual benchmarks
Benchmark	Gemma 3 1B	GPT-5.1
LMArena Non-English	—	1431
LMArena Chinese	—	1495
LMArena French	—	1450
LMArena German	—	1438
LMArena Japanese	—	1453
LMArena Korean	—	1401
LMArena Russian	—	1435
LMArena Spanish	—	1433

Instruction Following Not comparable

Gemma 3 1B: —, GPT-5.1: 83.9 (#1)

Instruction Following benchmarks
Benchmark	Gemma 3 1B	GPT-5.1
LiveBench Instruction Following	—	93.3%
IFEval	—	93.5%
LMArena Instruction Following	—	1443

Long Context Not comparable

Gemma 3 1B: —, GPT-5.1: 47.6 (#14)

Long Context benchmarks
Benchmark	Gemma 3 1B	GPT-5.1
CL-bench	—	23.7%
CL-bench Life	—	17.3%
LMArena Longer Query	—	1447

Writing & Preference Not comparable

Gemma 3 1B: —, GPT-5.1: 64.5 (#55)

Writing & Preference benchmarks
Benchmark	Gemma 3 1B	GPT-5.1
LMArena Text	—	1443
LMArena Creative Writing	—	1427
WildBench	—	86.3%
LMArena Multi-Turn	—	1450
LiveBench Language	—	80.2%

Frequently asked questions

Is Gemma 3 1B better than GPT-5.1?