Noometry

Model comparison

Gemma 3 1B vs GPT-4.1 mini

GPT-4.1 mini is the stronger model overall, scoring 33.6 to 21.1 on the Noometry Index.

Last verified . 4 shared benchmarks.

Gemma 3 1B Google

21.1

Rank #353 Reported

GPT-4.1 mini OpenAI

33.6

Rank #240 Confirmed

Summary

  • They share 4 benchmarks with published results for both. Gemma 3 1B scores higher in 1 category and GPT-4.1 mini in 3 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where GPT-4.1 mini leads 34.7 to 7.0.
  • The biggest single-benchmark swing is GPQA Diamond: 19.9% for Gemma 3 1B and 65.8% for GPT-4.1 mini.
  • Gemma 3 1B has downloadable open weights; the other is API-only.

Side by side

Gemma 3 1BGPT-4.1 mini
ProviderGoogleOpenAI
Noometry Index21.133.6
Released2025-03-122025-04-14
WeightsOpenProprietary
Context window—1.05M
Max output—33K
Input $ / M tokens—$0.40
Output $ / M tokens—$1.60
Results tracked447

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Gemma 3 1B: —, GPT-4.1 mini: 30.6 (#293)

BenchmarkGemma 3 1BGPT-4.1 mini
SWE-bench Verified (bash only)—23.9%
Aider Polyglot—32.4%
SciCode—40.4%
WeirdML—37.6%
BigCodeBench Instruct—48.9%
LMArena Coding—1367
CadEval—16%

Agentic & Tool Use GPT-4.1 mini leads

Gemma 3 1B: 13.7 (#152), GPT-4.1 mini: 33.3 (#55)

BenchmarkGemma 3 1BGPT-4.1 mini
Berkeley Function Calling Leaderboard7.2%50.5%

Reasoning Gemma 3 1B leads

Gemma 3 1B: 19.2 (#264), GPT-4.1 mini: 10.8 (#340)

BenchmarkGemma 3 1BGPT-4.1 mini
Chess Puzzles0%7%
ARC-AGI-2—0%
Kagi LLM Benchmark—48.6%
ARC-AGI-1—3.5%
CritPt—0%
LMArena Hard Prompts—1349
Mystery Game Puzzles—7%
DTBench—68.8%
LMCA—21.1%
Epoch Capabilities Index—135.01

Math GPT-4.1 mini leads

Gemma 3 1B: 10.2 (#316), GPT-4.1 mini: 24.1 (#270)

BenchmarkGemma 3 1BGPT-4.1 mini
OTIS Mock AIME 2024-20251.1%44.7%
FrontierMath (Tiers 1-3)—6.7%
Omni-MATH—49.1%
LMArena Math—1343
MATH Level 5—87.3%
FrontierMath (Feb 2025 set)—4.5%

Knowledge GPT-4.1 mini leads

Gemma 3 1B: 7.0 (#314), GPT-4.1 mini: 34.7 (#194)

BenchmarkGemma 3 1BGPT-4.1 mini
GPQA Diamond19.9%65.8%
SimpleQA Verified—12.7%
MMLU-Pro—78.3%
GPQA (HELM)—61.4%
LMArena Expert—1338

Multimodal Not comparable

Gemma 3 1B: —, GPT-4.1 mini: 35.8 (#82)

BenchmarkGemma 3 1BGPT-4.1 mini
LMArena Vision—1181

Multilingual Not comparable

Gemma 3 1B: —, GPT-4.1 mini: 45.7 (#166)

BenchmarkGemma 3 1BGPT-4.1 mini
LMArena Non-English—1318
LMArena Chinese—1329
LMArena French—1358
LMArena German—1351
LMArena Japanese—1290
LMArena Korean—1298
LMArena Russian—1324
LMArena Spanish—1319

Instruction Following Not comparable

Gemma 3 1B: —, GPT-4.1 mini: 73.7 (#118)

BenchmarkGemma 3 1BGPT-4.1 mini
IFEval—90.4%
LMArena Instruction Following—1333

Long Context Not comparable

Gemma 3 1B: —, GPT-4.1 mini: 31.8 (#275)

BenchmarkGemma 3 1BGPT-4.1 mini
Fiction.LiveBench—44.4%
LMArena Longer Query—1344

Writing & Preference Not comparable

Gemma 3 1B: —, GPT-4.1 mini: 48.6 (#199)

BenchmarkGemma 3 1BGPT-4.1 mini
LMArena Text—1340
LMArena Creative Writing—1300
EQ-Bench Creative Writing—1147
WildBench—83.8%
LMArena Multi-Turn—1354

Frequently asked questions

Is Gemma 3 1B better than GPT-4.1 mini?

GPT-4.1 mini is the stronger model overall, scoring 33.6 to 21.1 on the Noometry Index.

How many benchmarks do Gemma 3 1B and GPT-4.1 mini share?

4 benchmarks have published results for both models. Gemma 3 1B has 4 scored results on Noometry and GPT-4.1 mini has 47.

Related comparisons

Go deeper