Noometry

Model comparison

Gemma 2 9B vs Grok 4.1 Fast

Grok 4.1 Fast is the stronger model overall, scoring 41.4 to 25.9 on the Noometry Index.

Last verified . 18 shared benchmarks.

Gemma 2 9B Google

25.9

Rank #341 Confirmed

Grok 4.1 Fast xAI

41.4

Rank #136 Confirmed

Summary

  • They share 18 benchmarks with published results for both. Gemma 2 9B scores higher in 0 categories and Grok 4.1 Fast in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Grok 4.1 Fast leads 43.4 to 15.9.
  • Gemma 2 9B has downloadable open weights; the other is API-only.

Side by side

Gemma 2 9BGrok 4.1 Fast
ProviderGooglexAI
Noometry Index25.941.4
Released2024-06-242025-06-27
WeightsOpenProprietary
Context window—128K
Max output—30K
Input $ / M tokens—$0.20
Output $ / M tokens—$0.50
Results tracked3532

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok 4.1 Fast leads

Gemma 2 9B: 29.4 (#304), Grok 4.1 Fast: 34.1 (#245)

BenchmarkGemma 2 9BGrok 4.1 Fast
LMArena Coding11731411
LMArena WebDev—1242
BigCodeBench Instruct34.7%—
LiveBench Coding22.5%—
BigCodeBench Complete40.6%—
ALE-Bench—394.93

Agentic & Tool Use Not comparable

Gemma 2 9B: —, Grok 4.1 Fast: 36.3 (#39)

BenchmarkGemma 2 9BGrok 4.1 Fast
Berkeley Function Calling Leaderboard—69.6%
τ²-bench Banking—13.1%
LMArena Search—1171
Vending-Bench 2—1,107

Reasoning Grok 4.1 Fast leads

Gemma 2 9B: 15.9 (#309), Grok 4.1 Fast: 43.4 (#49)

BenchmarkGemma 2 9BGrok 4.1 Fast
LMArena Hard Prompts11711407
SimpleBench—56%
NYT Connections (extended)—87.4%
LiveBench Reasoning15.2%—
DTBench—87.7%
LiveBench Data Analysis36.4%—
Epoch Capabilities Index119.83—
ForecastBench—61
LiveBench28.7%—
PIQA83.7%—

Math Grok 4.1 Fast leads

Gemma 2 9B: 9.9 (#318), Grok 4.1 Fast: 31.9 (#221)

BenchmarkGemma 2 9BGrok 4.1 Fast
LMArena Math11831408
MathArena Final-Answer Competitions—60.9%
OTIS Mock AIME 2024-20250.6%—
ProofBench—4%
LiveBench Math19.8%—
MATH Level 521%—
GSM8K84.9%—

Knowledge Grok 4.1 Fast leads

Gemma 2 9B: 9.7 (#305), Grok 4.1 Fast: 33.1 (#207)

BenchmarkGemma 2 9BGrok 4.1 Fast
LMArena Expert11471399
GPQA Diamond27.5%—
Vectara Hallucination Rate—17.8%
BoolQ85.7%—
MMLU72.1%—

Multimodal Not comparable

Gemma 2 9B: —, Grok 4.1 Fast: 37.0 (#76)

BenchmarkGemma 2 9BGrok 4.1 Fast
LMArena Vision—1201

Multilingual Grok 4.1 Fast leads

Gemma 2 9B: 36.6 (#238), Grok 4.1 Fast: 51.0 (#114)

BenchmarkGemma 2 9BGrok 4.1 Fast
LMArena Non-English11881391
LMArena Chinese11851441
LMArena French11901415
LMArena German11861404
LMArena Japanese11441349
LMArena Korean11371361
LMArena Russian12001387
LMArena Spanish12001413

Instruction Following Grok 4.1 Fast leads

Gemma 2 9B: 57.6 (#269), Grok 4.1 Fast: 72.7 (#133)

BenchmarkGemma 2 9BGrok 4.1 Fast
LMArena Instruction Following11781376
LiveBench Instruction Following52.6%—

Long Context Grok 4.1 Fast leads

Gemma 2 9B: 36.3 (#233), Grok 4.1 Fast: 42.4 (#126)

BenchmarkGemma 2 9BGrok 4.1 Fast
LMArena Longer Query11971390

Writing & Preference Grok 4.1 Fast leads

Gemma 2 9B: 32.1 (#281), Grok 4.1 Fast: 57.2 (#131)

BenchmarkGemma 2 9BGrok 4.1 Fast
LMArena Text12071408
LMArena Creative Writing12061394
EQ-Bench Creative Writing8411327
LMArena Multi-Turn11931389
LiveBench Language25.5%—

Frequently asked questions

Is Gemma 2 9B better than Grok 4.1 Fast?

Grok 4.1 Fast is the stronger model overall, scoring 41.4 to 25.9 on the Noometry Index.

Is Gemma 2 9B or Grok 4.1 Fast better for coding?

Grok 4.1 Fast scores higher on coding benchmarks: 34.1 versus 29.4 in the Noometry coding category.

How many benchmarks do Gemma 2 9B and Grok 4.1 Fast share?

18 benchmarks have published results for both models. Gemma 2 9B has 35 scored results on Noometry and Grok 4.1 Fast has 32.

Related comparisons

Go deeper