Model comparison

Gemma 3 1B vs GPT-4.1

GPT-4.1 is the stronger model overall, scoring 35.9 to 21.1 on the Noometry Index.

Last verified October 10, 2026. 4 shared benchmarks.

Gemma 3 1B

Google

21.1

Rank #353 Reported

GPT-4.1 OpenAI

35.9

Rank #219 Confirmed

Summary

They share 4 benchmarks with published results for both. Gemma 3 1B scores higher in 1 category and GPT-4.1 in 3 categories; 4 gaps are clear of the uncertainty.
The widest gap is in knowledge, where GPT-4.1 leads 37.1 to 7.0.
The biggest single-benchmark swing is GPQA Diamond: 19.9% for Gemma 3 1B and 66.9% for GPT-4.1.
Gemma 3 1B has downloadable open weights; the other is API-only.

Side by side

Gemma 3 1B and GPT-4.1 specifications
	Gemma 3 1B	GPT-4.1
Provider	Google	OpenAI
Noometry Index	21.1	35.9
Released	2025-03-12	2025-04-14
Weights	Open	Proprietary
Context window	—	1.05M
Max output	—	33K
Input $ / M tokens	—	$2
Output $ / M tokens	—	$8
Results tracked	4	52

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Gemma 3 1B: —, GPT-4.1: 34.4 (#238)

Coding benchmarks
Benchmark	Gemma 3 1B	GPT-4.1
SWE-bench Verified	—	48.5%
SWE-bench Verified (bash only)	—	39.6%
Aider Polyglot	—	52.4%
WeirdML	—	39%
LMArena Coding	—	1391
CadEval	—	42%
ALE-Bench	—	558.1

Agentic & Tool Use GPT-4.1 leads

Gemma 3 1B: 13.7 (#152), GPT-4.1: 34.7 (#43)

Agentic & Tool Use benchmarks
Benchmark	Gemma 3 1B	GPT-4.1
Berkeley Function Calling Leaderboard	7.2%	54%

Reasoning Gemma 3 1B leads

Gemma 3 1B: 19.2 (#264), GPT-4.1: 11.7 (#339)

Reasoning benchmarks
Benchmark	Gemma 3 1B	GPT-4.1
Chess Puzzles	0%	6%
ARC-AGI-2	—	0.4%
SimpleBench	—	27%
Kagi LLM Benchmark	—	52.3%
ARC-AGI-1	—	5.5%
EnigmaEval	—	2.2%
LMArena Hard Prompts	—	1384
DTBench	—	68.3%
LMCA	—	25.6%
Epoch Capabilities Index	—	136.78
ForecastBench	—	61.5

Math GPT-4.1 leads

Gemma 3 1B: 10.2 (#316), GPT-4.1: 22.3 (#280)

Math benchmarks
Benchmark	Gemma 3 1B	GPT-4.1
OTIS Mock AIME 2024-2025	1.1%	38.3%
FrontierMath (Tiers 1-3)	—	6%
Omni-MATH	—	47.1%
LMArena Math	—	1370
MATH Level 5	—	83%
FrontierMath (Feb 2025 set)	—	5.5%
FrontierMath Tier 4 (v1)	—	0%

Knowledge GPT-4.1 leads

Gemma 3 1B: 7.0 (#314), GPT-4.1: 37.1 (#160)

Knowledge benchmarks
Benchmark	Gemma 3 1B	GPT-4.1
GPQA Diamond	19.9%	66.9%
Humanity's Last Exam	—	5.4%
SimpleQA Verified	—	31.1%
MMLU-Pro	—	81.1%
Vectara Hallucination Rate	—	5.6%
GPQA (HELM)	—	65.9%
LMArena Expert	—	1364

Multimodal Not comparable

Gemma 3 1B: —, GPT-4.1: 38.2 (#67)

Multimodal benchmarks
Benchmark	Gemma 3 1B	GPT-4.1
LMArena Vision	—	1211
GeoBench	—	72%

Multilingual Not comparable

Gemma 3 1B: —, GPT-4.1: 49.4 (#133)

Multilingual benchmarks
Benchmark	Gemma 3 1B	GPT-4.1
LMArena Non-English	—	1370
LMArena Chinese	—	1382
LMArena French	—	1382
LMArena German	—	1381
LMArena Japanese	—	1319
LMArena Korean	—	1339
LMArena Russian	—	1377
LMArena Spanish	—	1376

Instruction Following Not comparable

Gemma 3 1B: —, GPT-4.1: 71.3 (#153)

Instruction Following benchmarks
Benchmark	Gemma 3 1B	GPT-4.1
IFEval	—	83.8%
LMArena Instruction Following	—	1367

Long Context Not comparable

Gemma 3 1B: —, GPT-4.1: 40.0 (#163)

Long Context benchmarks
Benchmark	Gemma 3 1B	GPT-4.1
Fiction.LiveBench	—	63.9%
LMArena Longer Query	—	1385

Writing & Preference Not comparable

Gemma 3 1B: —, GPT-4.1: 57.6 (#125)

Writing & Preference benchmarks
Benchmark	Gemma 3 1B	GPT-4.1
LMArena Text	—	1383
LMArena Creative Writing	—	1363
EQ-Bench Creative Writing	—	1420
WildBench	—	85.4%
LMArena Multi-Turn	—	1398

Frequently asked questions

Is Gemma 3 1B better than GPT-4.1?