Noometry

Model comparison

Gemma 3 12B vs gpt-oss-20b

Gemma 3 12B and gpt-oss-20b score almost the same on the Noometry Index (32.1 vs 32.5), so choose on price, context window or the category you care about most.

Last verified . 21 shared benchmarks.

Gemma 3 12B Google

32.1

Rank #262 Confirmed

gpt-oss-20b OpenAI

32.5

Rank #255 Confirmed

Summary

  • They share 21 benchmarks with published results for both. Gemma 3 12B scores higher in 5 categories and gpt-oss-20b in 4 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in math, where gpt-oss-20b leads 39.4 to 22.3.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 16.7% for Gemma 3 12B and 65.3% for gpt-oss-20b.
  • gpt-oss-20b is cheaper at $0.018 / $0.09 per million input/output tokens, against $0.05 / $0.15 for Gemma 3 12B.

Side by side

Gemma 3 12Bgpt-oss-20b
ProviderGoogleOpenAI
Noometry Index32.132.5
Released2025-03-122025-08-05
WeightsOpenOpen
Context window131K131K
Max output8K16K
Input $ / M tokens$0.05$0.018
Output $ / M tokens$0.15$0.09
Results tracked2434

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding gpt-oss-20b leads

Gemma 3 12B: 31.7 (#280), gpt-oss-20b: 37.6 (#192)

BenchmarkGemma 3 12Bgpt-oss-20b
SciCode17.4%34.4%
LMArena Coding12811306
WeirdML—40.9%
ALE-Bench—566.05

Agentic & Tool Use Gemma 3 12B leads

Gemma 3 12B: 25.5 (#108), gpt-oss-20b: 9.3 (#154)

BenchmarkGemma 3 12Bgpt-oss-20b
Terminal-Bench—3.4%
Berkeley Function Calling Leaderboard30.4%—

Reasoning gpt-oss-20b leads

Gemma 3 12B: 15.7 (#313), gpt-oss-20b: 19.3 (#261)

BenchmarkGemma 3 12Bgpt-oss-20b
CritPt0%1.4%
Chess Puzzles0%4%
LMArena Hard Prompts13091274
DTBench48.8%68%
LMCA4.5%14.5%
Epoch Capabilities Index123.5137.82
Kagi LLM Benchmark—53.2%

Math gpt-oss-20b leads

Gemma 3 12B: 22.3 (#279), gpt-oss-20b: 39.4 (#103)

BenchmarkGemma 3 12Bgpt-oss-20b
OTIS Mock AIME 2024-202516.7%65.3%
LMArena Math13071317
Omni-MATH—56.5%

Knowledge gpt-oss-20b leads

Gemma 3 12B: 26.5 (#257), gpt-oss-20b: 34.6 (#195)

BenchmarkGemma 3 12Bgpt-oss-20b
GPQA Diamond39.5%60.8%
LMArena Expert12481258
MMLU-Pro—74%
Vectara Hallucination Rate4.4%—
GPQA (HELM)—59.4%

Multimodal Not comparable

Gemma 3 12B: —, gpt-oss-20b: —

BenchmarkGemma 3 12Bgpt-oss-20b
MindCube46.7%—

Multilingual Gemma 3 12B leads

Gemma 3 12B: 45.7 (#165), gpt-oss-20b: 42.2 (#197)

BenchmarkGemma 3 12Bgpt-oss-20b
LMArena Non-English13181268
LMArena German13701255
LMArena Russian13351278
LMArena Chinese—1314
LMArena Japanese—1244
LMArena Korean—1236
LMArena Spanish—1267

Instruction Following Gemma 3 12B leads

Gemma 3 12B: 68.6 (#186), gpt-oss-20b: 61.8 (#240)

BenchmarkGemma 3 12Bgpt-oss-20b
LMArena Instruction Following12991236
IFEval—73.2%

Long Context Gemma 3 12B leads

Gemma 3 12B: 40.0 (#162), gpt-oss-20b: 37.9 (#209)

BenchmarkGemma 3 12Bgpt-oss-20b
LMArena Longer Query13171250

Writing & Preference Gemma 3 12B leads

Gemma 3 12B: 47.5 (#209), gpt-oss-20b: 35.5 (#265)

BenchmarkGemma 3 12Bgpt-oss-20b
LMArena Text13341287
LMArena Creative Writing13311201
EQ-Bench Creative Writing1126666
LMArena Multi-Turn13341268
WildBench—73.7%

Frequently asked questions

Is Gemma 3 12B better than gpt-oss-20b?

Gemma 3 12B and gpt-oss-20b score almost the same on the Noometry Index (32.1 vs 32.5), so choose on price, context window or the category you care about most.

Which is cheaper, Gemma 3 12B or gpt-oss-20b?

gpt-oss-20b is cheaper. It lists at $0.018 per million input tokens and $0.09 per million output tokens; Gemma 3 12B lists at $0.05 and $0.15.

Is Gemma 3 12B or gpt-oss-20b better for coding?

gpt-oss-20b scores higher on coding benchmarks: 37.6 versus 31.7 in the Noometry coding category.

Which has the bigger context window?

Both accept 131K tokens.

How many benchmarks do Gemma 3 12B and gpt-oss-20b share?

21 benchmarks have published results for both models. Gemma 3 12B has 24 scored results on Noometry and gpt-oss-20b has 34.

Related comparisons

Go deeper