Model comparison

Gemma 3n E4b IT vs o1

o1 is the stronger model overall, scoring 40.9 to 37.3 on the Noometry Index.

Last verified October 10, 2026. 17 shared benchmarks.

Gemma 3n E4b IT

Google

37.3

Rank #206 Confirmed

o1 OpenAI

40.9

Rank #143 Confirmed

Summary

They share 17 benchmarks with published results for both. Gemma 3n E4b IT scores higher in 0 categories and o1 in 8 categories; 8 gaps are clear of the uncertainty.
The widest gap is in long context, where o1 leads 50.3 to 38.7.
Gemma 3n E4b IT has downloadable open weights; the other is API-only.

Side by side

Gemma 3n E4b IT and o1 specifications
	Gemma 3n E4b IT	o1
Provider	Google	OpenAI
Noometry Index	37.3	40.9
Released	—	2024-09-12
Weights	Open	Proprietary
Context window	—	200K
Max output	—	100K
Input $ / M tokens	—	$15
Output $ / M tokens	—	$60
Results tracked	18	52

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o1 leads

Gemma 3n E4b IT: 37.0 (#198), o1: 46.1 (#70)

Coding benchmarks
Benchmark	Gemma 3n E4b IT	o1
LMArena Coding	1268	1367
Aider Polyglot	—	61.7%
WeirdML	—	47.6%
LiveBench Coding	—	69.7%
CadEval	—	56%
HumanEval+	—	89%
MBPP+	—	80.2%

Agentic & Tool Use Not comparable

Gemma 3n E4b IT: —, o1: 24.6 (#117)

Agentic & Tool Use benchmarks
Benchmark	Gemma 3n E4b IT	o1
Cybench	—	10%
METR Time Horizons	—	51.1%

Reasoning o1 leads

Gemma 3n E4b IT: 19.9 (#247), o1: 27.9 (#111)

Reasoning benchmarks
Benchmark	Gemma 3n E4b IT	o1
LMArena Hard Prompts	1284	1371
SimpleBench	—	41.7%
Kagi LLM Benchmark	31.5%	—
ARC-AGI-1	—	30.7%
Chess Puzzles	—	15%
EnigmaEval	—	5.7%
LiveBench Reasoning	—	91.6%
DTBench	—	74.7%
LiveBench Data Analysis	—	65.5%
LMCA	—	22.3%
Epoch Capabilities Index	—	141.91
LiveBench	—	75.7%

Math o1 leads

Gemma 3n E4b IT: 35.1 (#188), o1: 36.1 (#175)

Math benchmarks
Benchmark	Gemma 3n E4b IT	o1
LMArena Math	1251	1388
FrontierMath (Tiers 1-3)	—	14.7%
OTIS Mock AIME 2024-2025	—	73.3%
LiveBench Math	—	80.3%
MATH Level 5	—	94.7%
FrontierMath (Feb 2025 set)	—	9.3%

Knowledge o1 leads

Gemma 3n E4b IT: 34.2 (#198), o1: 41.5 (#110)

Knowledge benchmarks
Benchmark	Gemma 3n E4b IT	o1
LMArena Expert	1246	1361
GPQA Diamond	—	76.8%
Humanity's Last Exam	—	8%
SimpleQA Verified	—	41.1%
Confabulations	—	11.7%

Multimodal Not comparable

Gemma 3n E4b IT: —, o1: 34.2 (#93)

Multimodal benchmarks
Benchmark	Gemma 3n E4b IT	o1
LMArena Vision	—	1168
GeoBench	—	80%
VPCT	—	37%
SpatialViz-Bench	—	41.4%

Multilingual o1 leads

Gemma 3n E4b IT: 43.4 (#183), o1: 48.6 (#142)

Multilingual benchmarks
Benchmark	Gemma 3n E4b IT	o1
LMArena Non-English	1285	1358
LMArena Chinese	1309	1394
LMArena French	1330	1344
LMArena German	1311	1337
LMArena Japanese	1272	1346
LMArena Korean	1259	1396
LMArena Russian	1288	1356
LMArena Spanish	1305	1345

Instruction Following o1 leads

Gemma 3n E4b IT: 66.1 (#210), o1: 74.8 (#86)

Instruction Following benchmarks
Benchmark	Gemma 3n E4b IT	o1
LMArena Instruction Following	1255	1367
LiveBench Instruction Following	—	81.5%

Long Context o1 leads

Gemma 3n E4b IT: 38.7 (#191), o1: 50.3 (#9)

Long Context benchmarks
Benchmark	Gemma 3n E4b IT	o1
LMArena Longer Query	1276	1378
Fiction.LiveBench	—	83.3%

Writing & Preference o1 leads

Gemma 3n E4b IT: 50.1 (#186), o1: 55.6 (#144)

Writing & Preference benchmarks
Benchmark	Gemma 3n E4b IT	o1
LMArena Text	1306	1366
LMArena Creative Writing	1287	1348
LMArena Multi-Turn	1276	1369
Short-Story Creative Writing	—	70.2%
LiveBench Language	—	65.4%

Frequently asked questions

Is Gemma 3n E4b IT better than o1?