Model comparison

Gemma 3 1B vs GPT-4o

GPT-4o is the stronger model overall, scoring 28.6 to 21.1 on the Noometry Index.

Last verified October 10, 2026. 3 shared benchmarks.

Gemma 3 1B

Google

21.1

Rank #353 Reported

GPT-4o OpenAI

28.6

Rank #324 Confirmed

Summary

They share 3 benchmarks with published results for both. Gemma 3 1B scores higher in 1 category and GPT-4o in 3 categories; 3 gaps are clear of the uncertainty.
The widest gap is in knowledge, where GPT-4o leads 28.8 to 7.0.
The biggest single-benchmark swing is GPQA Diamond: 19.9% for Gemma 3 1B and 49.2% for GPT-4o.
Gemma 3 1B has downloadable open weights; the other is API-only.

Side by side

Gemma 3 1B and GPT-4o specifications
	Gemma 3 1B	GPT-4o
Provider	Google	OpenAI
Noometry Index	21.1	28.6
Released	2025-03-12	2024-05-13
Weights	Open	Proprietary
Context window	—	128K
Max output	—	16K
Input $ / M tokens	—	$2.50
Output $ / M tokens	—	$10
Results tracked	4	72

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Gemma 3 1B: —, GPT-4o: 24.8 (#328)

Coding benchmarks
Benchmark	Gemma 3 1B	GPT-4o
SWE-bench Verified	—	31%
SWE-bench Verified (bash only)	—	21.6%
Aider Polyglot	—	45.3%
GSO	—	0%
WeirdML	—	25.1%
BigCodeBench Instruct	—	51.1%
LiveBench Coding	—	51.4%
LMArena Coding	—	1297
BigCodeBench Complete	—	61.1%
CadEval	—	26%
HumanEval+	—	87.2%
MBPP+	—	72.2%

Agentic & Tool Use GPT-4o leads

Gemma 3 1B: 13.7 (#152), GPT-4o: 21.0 (#141)

Agentic & Tool Use benchmarks
Benchmark	Gemma 3 1B	GPT-4o
Berkeley Function Calling Leaderboard	7.2%	—
GDPval	—	9.9%
TheAgentCompany	—	8.6%
Cybench	—	12.5%
BALROG	—	32.3%
LMArena Search	—	1006
METR Time Horizons	—	40.8%

Reasoning Gemma 3 1B leads

Gemma 3 1B: 19.2 (#264), GPT-4o: 9.4 (#343)

Reasoning benchmarks
Benchmark	Gemma 3 1B	GPT-4o
Chess Puzzles	0%	13%
ARC-AGI-2	—	0%
SimpleBench	—	17.8%
ARC-AGI-1	—	4.5%
CritPt	—	0%
EnigmaEval	—	0.8%
LiveBench Reasoning	—	55.8%
LMArena Hard Prompts	—	1281
DTBench	—	64.5%
LiveBench Data Analysis	—	60.9%
LMCA	—	16.6%
Epoch Capabilities Index	—	128.97
ForecastBench	—	57.7
LiveBench	—	55.3%

Math Too close to call

Gemma 3 1B: 10.2 (#316), GPT-4o: 10.6 (#312)

Math benchmarks
Benchmark	Gemma 3 1B	GPT-4o
OTIS Mock AIME 2024-2025	1.1%	6.4%
FrontierMath (Tiers 1-3)	—	0.4%
Omni-MATH	—	29.3%
LiveBench Math	—	49.5%
LMArena Math	—	1285
MATH Level 5	—	53.3%
FrontierMath (Feb 2025 set)	—	0.3%

Knowledge GPT-4o leads

Gemma 3 1B: 7.0 (#314), GPT-4o: 28.8 (#242)

Knowledge benchmarks
Benchmark	Gemma 3 1B	GPT-4o
GPQA Diamond	19.9%	49.2%
Humanity's Last Exam	—	2.7%
SimpleQA Verified	—	26%
MMLU-Pro	—	71.3%
Confabulations	—	15.3%
Vectara Hallucination Rate	—	9.6%
GPQA (HELM)	—	52%
LMArena Expert	—	1250
MMLU	—	88.1%

Multimodal Not comparable

Gemma 3 1B: —, GPT-4o: 34.5 (#91)

Multimodal benchmarks
Benchmark	Gemma 3 1B	GPT-4o
LMArena Vision	—	1137
Video-MME	—	71.9%
GeoBench	—	71%
VPCT	—	40%
ScienceQA	—	88.5%

Multilingual Not comparable

Gemma 3 1B: —, GPT-4o: 43.2 (#186)

Multilingual benchmarks
Benchmark	Gemma 3 1B	GPT-4o
LMArena Non-English	—	1283
LMArena Chinese	—	1277
LMArena French	—	1304
LMArena German	—	1282
LMArena Japanese	—	1257
LMArena Korean	—	1234
LMArena Russian	—	1286
LMArena Spanish	—	1292

Instruction Following Not comparable

Gemma 3 1B: —, GPT-4o: 66.6 (#207)

Instruction Following benchmarks
Benchmark	Gemma 3 1B	GPT-4o
LiveBench Instruction Following	—	68.6%
IFEval	—	81.7%
LMArena Instruction Following	—	1278

Long Context Not comparable

Gemma 3 1B: —, GPT-4o: 39.4 (#179)

Long Context benchmarks
Benchmark	Gemma 3 1B	GPT-4o
Fiction.LiveBench	—	66.7%
LMArena Longer Query	—	1289

Writing & Preference Not comparable

Gemma 3 1B: —, GPT-4o: 52.6 (#166)

Writing & Preference benchmarks
Benchmark	Gemma 3 1B	GPT-4o
LMArena Text	—	1300
LMArena Creative Writing	—	1292
Short-Story Creative Writing	—	81.8%
WildBench	—	82.8%
LMArena Multi-Turn	—	1302
LiveBench Language	—	47.6%

Frequently asked questions

Is Gemma 3 1B better than GPT-4o?