Noometry

Model comparison

Gemma 3 1B vs GPT-5 Mini

GPT-5 Mini is the stronger model overall, scoring 41.8 to 21.1 on the Noometry Index.

Last verified . 4 shared benchmarks.

Gemma 3 1B Google

21.1

Rank #353 Reported

GPT-5 Mini OpenAI

41.8

Rank #128 Confirmed

Summary

  • They share 4 benchmarks with published results for both. Gemma 3 1B scores higher in 0 categories and GPT-5 Mini in 4 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where GPT-5 Mini leads 45.6 to 7.0.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 1.1% for Gemma 3 1B and 86.7% for GPT-5 Mini.
  • Gemma 3 1B has downloadable open weights; the other is API-only.

Side by side

Gemma 3 1BGPT-5 Mini
ProviderGoogleOpenAI
Noometry Index21.141.8
Released2025-03-122025-08-07
WeightsOpenProprietary
Context window—400K
Max output—128K
Input $ / M tokens—$0.25
Output $ / M tokens—$2
Results tracked460

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Gemma 3 1B: —, GPT-5 Mini: 40.1 (#146)

BenchmarkGemma 3 1BGPT-5 Mini
SWE-bench Verified—64.7%
SWE-bench Verified (bash only)—59.8%
SWE-bench Multilingual—39.7%
SciCode—39.2%
WeirdML—52.7%
LMArena Coding—1406
ALE-Bench—799.77
AlgoTune—1.38

Agentic & Tool Use GPT-5 Mini leads

Gemma 3 1B: 13.7 (#152), GPT-5 Mini: 31.1 (#70)

BenchmarkGemma 3 1BGPT-5 Mini
Berkeley Function Calling Leaderboard7.2%55.5%
Terminal-Bench—34.8%
Vending-Bench 2—-31.18

Reasoning GPT-5 Mini leads

Gemma 3 1B: 19.2 (#264), GPT-5 Mini: 23.9 (#168)

BenchmarkGemma 3 1BGPT-5 Mini
Chess Puzzles0%30%
ARC-AGI-2—4.4%
Kagi LLM Benchmark—70.3%
ARC-AGI-1—54.3%
CritPt—0%
EnigmaEval—8.2%
LMArena Hard Prompts—1380
Mystery Game Puzzles—10%
DTBench—80.5%
LMCA—34.2%
Epoch Capabilities Index—145.52
ForecastBench—61

Math GPT-5 Mini leads

Gemma 3 1B: 10.2 (#316), GPT-5 Mini: 46.7 (#69)

BenchmarkGemma 3 1BGPT-5 Mini
OTIS Mock AIME 2024-20251.1%86.7%
FrontierMath (Tiers 1-3)—46.7%
FrontierMath Tier 4—12.2%
ProofBench—9%
Omni-MATH—72.2%
LMArena Math—1378
MATH Level 5—97.8%
FrontierMath (Feb 2025 set)—27.2%
FrontierMath Tier 4 (v1)—6.3%

Knowledge GPT-5 Mini leads

Gemma 3 1B: 7.0 (#314), GPT-5 Mini: 45.6 (#86)

BenchmarkGemma 3 1BGPT-5 Mini
GPQA Diamond19.9%75%
Humanity's Last Exam—19.4%
SimpleQA Verified—21.6%
MMLU-Pro—83.5%
Confabulations—13.3%
Vectara Hallucination Rate—12.9%
GPQA (HELM)—75.6%
LMArena Expert—1379

Multimodal Not comparable

Gemma 3 1B: —, GPT-5 Mini: 35.6 (#85)

BenchmarkGemma 3 1BGPT-5 Mini
LMArena Vision—1202
VPCT—40.2%

Multilingual Not comparable

Gemma 3 1B: —, GPT-5 Mini: 48.9 (#137)

BenchmarkGemma 3 1BGPT-5 Mini
LMArena Non-English—1363
LMArena Chinese—1385
LMArena French—1386
LMArena German—1366
LMArena Japanese—1341
LMArena Korean—1308
LMArena Russian—1362
LMArena Spanish—1355

Instruction Following Not comparable

Gemma 3 1B: —, GPT-5 Mini: 76.2 (#46)

BenchmarkGemma 3 1BGPT-5 Mini
IFEval—92.7%
LMArena Instruction Following—1357

Long Context Not comparable

Gemma 3 1B: —, GPT-5 Mini: 41.9 (#132)

BenchmarkGemma 3 1BGPT-5 Mini
Fiction.LiveBench—69.4%
LMArena Longer Query—1355

Writing & Preference Not comparable

Gemma 3 1B: —, GPT-5 Mini: 55.2 (#148)

BenchmarkGemma 3 1BGPT-5 Mini
LMArena Text—1373
LMArena Creative Writing—1325
Short-Story Creative Writing—83.1%
EQ-Bench Creative Writing—1313
WildBench—85.5%
LMArena Multi-Turn—1363

Frequently asked questions

Is Gemma 3 1B better than GPT-5 Mini?

GPT-5 Mini is the stronger model overall, scoring 41.8 to 21.1 on the Noometry Index.

How many benchmarks do Gemma 3 1B and GPT-5 Mini share?

4 benchmarks have published results for both models. Gemma 3 1B has 4 scored results on Noometry and GPT-5 Mini has 60.

Related comparisons

Go deeper