Noometry

Model comparison

Gemma 2 27B vs Grok 4

Grok 4 is the stronger model overall, scoring 47.8 to 29.2 on the Noometry Index.

Last verified . 21 shared benchmarks.

Gemma 2 27B Google

29.2

Rank #302 Confirmed

Grok 4 xAI

47.8

Rank #49 Confirmed

Summary

  • They share 21 benchmarks with published results for both. Gemma 2 27B scores higher in 0 categories and Grok 4 in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Grok 4 leads 46.5 to 10.0.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 1.4% for Gemma 2 27B and 84% for Grok 4.
  • Gemma 2 27B has downloadable open weights; the other is API-only.

Side by side

Gemma 2 27BGrok 4
ProviderGooglexAI
Noometry Index29.247.8
Released2024-06-242025-07-09
WeightsOpenProprietary
Context window8K—
Max output2K—
Input $ / M tokens$0.65—
Output $ / M tokens$0.65—
Results tracked3448

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok 4 leads

Gemma 2 27B: 33.5 (#243), Grok 4: 49.9 (#44)

BenchmarkGemma 2 27BGrok 4
LMArena Coding12111408
Aider Polyglot—79.6%
WeirdML—45.7%
BigCodeBench Instruct42.8%—
LiveBench Coding36%—
BigCodeBench Complete52.5%—

Agentic & Tool Use Not comparable

Gemma 2 27B: —, Grok 4: 32.1 (#62)

BenchmarkGemma 2 27BGrok 4
Terminal-Bench—27.2%
Berkeley Function Calling Leaderboard—63%
GDPval—21.1%
Cybench—43%
DeepResearch Bench—47.3%
BALROG—43.6%
LMArena Search—1142
METR Time Horizons—66.6%

Reasoning Grok 4 leads

Gemma 2 27B: 14.8 (#313), Grok 4: 36.9 (#59)

BenchmarkGemma 2 27BGrok 4
LMArena Hard Prompts11991409
Epoch Capabilities Index122.06146.46
ARC-AGI-2—16%
SimpleBench—60.5%
Kagi LLM Benchmark—73.6%
ARC-AGI-1—66.7%
Chess Puzzles—28%
LiveBench Reasoning28.1%—
DTBench48%—
LiveBench Data Analysis47.9%—
LMCA7.1%—
ForecastBench—60.9
LiveBench38.2%—

Math Grok 4 leads

Gemma 2 27B: 10.0 (#306), Grok 4: 46.5 (#58)

BenchmarkGemma 2 27BGrok 4
OTIS Mock AIME 2024-20251.4%84%
LMArena Math12121422
Omni-MATH—60.3%
LiveBench Math26.5%—
MATH Level 527.9%—
FrontierMath (Feb 2025 set)—19.7%
FrontierMath Tier 4 (v1)—2.1%

Knowledge Grok 4 leads

Gemma 2 27B: 18.9 (#275), Grok 4: 53.7 (#50)

BenchmarkGemma 2 27BGrok 4
GPQA Diamond36.5%87%
Confabulations27.1%12.4%
LMArena Expert11721417
MMLU-Pro—85.1%
GPQA (HELM)—72.7%
MMLU75.7%—

Multimodal Not comparable

Gemma 2 27B: —, Grok 4: 33.1 (#89)

BenchmarkGemma 2 27BGrok 4
LMArena Vision—1211
GeoBench—45%

Multilingual Grok 4 leads

Gemma 2 27B: 38.8 (#221), Grok 4: 52.0 (#98)

BenchmarkGemma 2 27BGrok 4
LMArena Non-English12171403
LMArena Chinese12211428
LMArena French12461418
LMArena German12091429
LMArena Japanese11751396
LMArena Korean11741376
LMArena Russian12341409
LMArena Spanish12281419

Instruction Following Grok 4 leads

Gemma 2 27B: 60.7 (#244), Grok 4: 79.4 (#4)

BenchmarkGemma 2 27BGrok 4
LMArena Instruction Following12061387
LiveBench Instruction Following58.1%—
IFEval—94.9%

Long Context Grok 4 leads

Gemma 2 27B: 37.3 (#213), Grok 4: 63.1 (#4)

BenchmarkGemma 2 27BGrok 4
LMArena Longer Query12311409
Fiction.LiveBench—94.4%

Writing & Preference Grok 4 leads

Gemma 2 27B: 44.4 (#220), Grok 4: 58.7 (#111)

BenchmarkGemma 2 27BGrok 4
LMArena Text12311412
LMArena Creative Writing12411397
LMArena Multi-Turn12241416
Short-Story Creative Writing—76.9%
WildBench—79.7%
LiveBench Language32.6%—

Frequently asked questions

Is Gemma 2 27B better than Grok 4?

Grok 4 is the stronger model overall, scoring 47.8 to 29.2 on the Noometry Index.

Is Gemma 2 27B or Grok 4 better for coding?

Grok 4 scores higher on coding benchmarks: 49.9 versus 33.5 in the Noometry coding category.

How many benchmarks do Gemma 2 27B and Grok 4 share?

21 benchmarks have published results for both models. Gemma 2 27B has 34 scored results on Noometry and Grok 4 has 48.

Related comparisons

Go deeper