Noometry

Model comparison

Gemma 2 9B vs Grok-3 mini

Grok-3 mini is the stronger model overall, scoring 41.0 to 25.7 on the Noometry Index.

Last verified . 21 shared benchmarks.

Gemma 2 9B Google

25.7

Rank #334 Confirmed

Grok-3 mini xAI

41.0

Rank #130 Confirmed

Summary

  • They share 21 benchmarks with published results for both. Gemma 2 9B scores higher in 1 category and Grok-3 mini in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Grok-3 mini leads 46.3 to 9.7.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 0.6% for Gemma 2 9B and 77.8% for Grok-3 mini.
  • Gemma 2 9B has downloadable open weights; the other is API-only.

Side by side

Gemma 2 9BGrok-3 mini
ProviderGooglexAI
Noometry Index25.741.0
Released2024-06-242025-04-09
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3535

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok-3 mini leads

Gemma 2 9B: 29.0 (#299), Grok-3 mini: 40.4 (#126)

BenchmarkGemma 2 9BGrok-3 mini
LMArena Coding11741380
Aider Polyglot—49.3%
WeirdML—42.6%
BigCodeBench Instruct34.7%—
LiveBench Coding22.5%—
BigCodeBench Complete40.6%—

Reasoning Gemma 2 9B leads

Gemma 2 9B: 15.3 (#306), Grok-3 mini: 13.8 (#323)

BenchmarkGemma 2 9BGrok-3 mini
LMArena Hard Prompts11721374
Epoch Capabilities Index119.78140.35
ARC-AGI-2—0.4%
Kagi LLM Benchmark—61.3%
ARC-AGI-1—16.5%
LiveBench Reasoning15.2%—
LiveBench Data Analysis36.4%—
LiveBench28.7%—
PIQA83.7%—

Math Grok-3 mini leads

Gemma 2 9B: 9.3 (#312), Grok-3 mini: 40.3 (#78)

BenchmarkGemma 2 9BGrok-3 mini
OTIS Mock AIME 2024-20250.6%77.8%
LMArena Math11831386
MATH Level 521%90.9%
Omni-MATH—31.8%
LiveBench Math19.8%—
FrontierMath (Feb 2025 set)—5.9%
GSM8K84.9%—

Knowledge Grok-3 mini leads

Gemma 2 9B: 9.7 (#300), Grok-3 mini: 46.3 (#75)

BenchmarkGemma 2 9BGrok-3 mini
GPQA Diamond27.5%76.3%
LMArena Expert11471395
MMLU-Pro—79.9%
Confabulations—10.8%
GPQA (HELM)—67.5%
BoolQ85.7%—
MMLU72.1%—

Multilingual Grok-3 mini leads

Gemma 2 9B: 36.8 (#233), Grok-3 mini: 48.3 (#141)

BenchmarkGemma 2 9BGrok-3 mini
LMArena Non-English11881352
LMArena Chinese11851387
LMArena French11891357
LMArena German11861350
LMArena Japanese11441343
LMArena Korean11371336
LMArena Russian12001352
LMArena Spanish12001379

Instruction Following Grok-3 mini leads

Gemma 2 9B: 57.8 (#264), Grok-3 mini: 78.7 (#8)

BenchmarkGemma 2 9BGrok-3 mini
LMArena Instruction Following11781356
LiveBench Instruction Following52.6%—
IFEval—95.1%

Long Context Grok-3 mini leads

Gemma 2 9B: 36.3 (#228), Grok-3 mini: 41.0 (#142)

BenchmarkGemma 2 9BGrok-3 mini
LMArena Longer Query11971372
Fiction.LiveBench—66.7%

Writing & Preference Grok-3 mini leads

Gemma 2 9B: 32.3 (#276), Grok-3 mini: 52.7 (#163)

BenchmarkGemma 2 9BGrok-3 mini
LMArena Text12071370
LMArena Creative Writing12061342
LMArena Multi-Turn11931355
Short-Story Creative Writing—73.5%
EQ-Bench Creative Writing841—
WildBench—65.1%
LiveBench Language25.5%—

Frequently asked questions

Is Gemma 2 9B better than Grok-3 mini?

Grok-3 mini is the stronger model overall, scoring 41.0 to 25.7 on the Noometry Index.

Is Gemma 2 9B or Grok-3 mini better for coding?

Grok-3 mini scores higher on coding benchmarks: 40.4 versus 29.0 in the Noometry coding category.

How many benchmarks do Gemma 2 9B and Grok-3 mini share?

21 benchmarks have published results for both models. Gemma 2 9B has 35 scored results on Noometry and Grok-3 mini has 35.

Related comparisons

Go deeper