Noometry

Model comparison

Gemma 2 9B vs GPT-4.5

GPT-4.5 is the stronger model overall, scoring 37.2 to 25.9 on the Noometry Index.

Last verified . 28 shared benchmarks.

Gemma 2 9B Google

25.9

Rank #341 Confirmed

GPT-4.5 OpenAI

37.2

Rank #208 Confirmed

Summary

  • They share 28 benchmarks with published results for both. Gemma 2 9B scores higher in 1 category and GPT-4.5 in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where GPT-4.5 leads 56.9 to 32.1.
  • The biggest single-benchmark swing is MATH Level 5: 21% for Gemma 2 9B and 78.6% for GPT-4.5.
  • Gemma 2 9B has downloadable open weights; the other is API-only.

Side by side

Gemma 2 9BGPT-4.5
ProviderGoogleOpenAI
Noometry Index25.937.2
Released2024-06-242025-02-27
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3542

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4.5 leads

Gemma 2 9B: 29.4 (#304), GPT-4.5: 42.2 (#109)

BenchmarkGemma 2 9BGPT-4.5
LiveBench Coding22.5%75.2%
LMArena Coding11731396
Aider Polyglot—44.9%
WeirdML—39.4%
BigCodeBench Instruct34.7%—
BigCodeBench Complete40.6%—

Agentic & Tool Use Not comparable

Gemma 2 9B: —, GPT-4.5: 27.9 (#97)

BenchmarkGemma 2 9BGPT-4.5
Cybench—17.5%

Reasoning Gemma 2 9B leads

Gemma 2 9B: 15.9 (#309), GPT-4.5: 13.9 (#330)

BenchmarkGemma 2 9BGPT-4.5
LiveBench Reasoning15.2%71.1%
LMArena Hard Prompts11711403
LiveBench Data Analysis36.4%64.3%
Epoch Capabilities Index119.83136.74
LiveBench28.7%69%
ARC-AGI-2—0.8%
SimpleBench—34.5%
ARC-AGI-1—10.3%
EnigmaEval—3.2%
ForecastBench—61.7
PIQA83.7%—

Math GPT-4.5 leads

Gemma 2 9B: 9.9 (#318), GPT-4.5: 32.6 (#211)

BenchmarkGemma 2 9BGPT-4.5
OTIS Mock AIME 2024-20250.6%37.8%
LiveBench Math19.8%69.3%
LMArena Math11831412
MATH Level 521%78.6%
GSM8K84.9%—

Knowledge GPT-4.5 leads

Gemma 2 9B: 9.7 (#305), GPT-4.5: 32.5 (#211)

BenchmarkGemma 2 9BGPT-4.5
GPQA Diamond27.5%68.7%
LMArena Expert11471394
Humanity's Last Exam—5.4%
Confabulations—13.6%
BoolQ85.7%—
MMLU72.1%—

Multimodal Not comparable

Gemma 2 9B: —, GPT-4.5: 37.6 (#71)

BenchmarkGemma 2 9BGPT-4.5
LMArena Vision—1195
VPCT—45%

Multilingual GPT-4.5 leads

Gemma 2 9B: 36.6 (#238), GPT-4.5: 52.5 (#83)

BenchmarkGemma 2 9BGPT-4.5
LMArena Non-English11881413
LMArena Chinese11851421
LMArena French11901418
LMArena German11861457
LMArena Japanese11441416
LMArena Korean11371392
LMArena Russian12001419
LMArena Spanish1200—

Instruction Following GPT-4.5 leads

Gemma 2 9B: 57.6 (#269), GPT-4.5: 72.6 (#134)

BenchmarkGemma 2 9BGPT-4.5
LiveBench Instruction Following52.6%72.3%
LMArena Instruction Following11781404

Long Context GPT-4.5 leads

Gemma 2 9B: 36.3 (#233), GPT-4.5: 40.4 (#155)

BenchmarkGemma 2 9BGPT-4.5
LMArena Longer Query11971406
Fiction.LiveBench—63.9%

Writing & Preference GPT-4.5 leads

Gemma 2 9B: 32.1 (#281), GPT-4.5: 56.9 (#134)

BenchmarkGemma 2 9BGPT-4.5
LMArena Text12071417
LMArena Creative Writing12061394
EQ-Bench Creative Writing8411258
LMArena Multi-Turn11931444
LiveBench Language25.5%61.5%
Short-Story Creative Writing—75.6%

Frequently asked questions

Is Gemma 2 9B better than GPT-4.5?

GPT-4.5 is the stronger model overall, scoring 37.2 to 25.9 on the Noometry Index.

Is Gemma 2 9B or GPT-4.5 better for coding?

GPT-4.5 scores higher on coding benchmarks: 42.2 versus 29.4 in the Noometry coding category.

How many benchmarks do Gemma 2 9B and GPT-4.5 share?

28 benchmarks have published results for both models. Gemma 2 9B has 35 scored results on Noometry and GPT-4.5 has 42.

Related comparisons

Go deeper