Noometry

Model comparison

Gemma 4 26B A4B IT vs Grok-3 mini

Gemma 4 26B A4B IT is the stronger model overall, scoring 43.1 to 41.0 on the Noometry Index.

Last verified . 18 shared benchmarks.

Gemma 4 26B A4B IT Google

43.1

Rank #88 Confirmed

Grok-3 mini xAI

41.0

Rank #130 Confirmed

Summary

  • They share 18 benchmarks with published results for both. Gemma 4 26B A4B IT scores higher in 5 categories and Grok-3 mini in 3 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Gemma 4 26B A4B IT leads 21.1 to 13.8.
  • The biggest single-benchmark swing is WeirdML: 35.2% for Gemma 4 26B A4B IT and 42.6% for Grok-3 mini.
  • Gemma 4 26B A4B IT has downloadable open weights; the other is API-only.

Side by side

Gemma 4 26B A4B ITGrok-3 mini
ProviderGooglexAI
Noometry Index43.141.0
Released2026-04-022025-04-09
WeightsOpenProprietary
Context window262K—
Max output33K—
Input $ / M tokens$0.0765—
Output $ / M tokens$0.26—
Results tracked2835

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok-3 mini leads

Gemma 4 26B A4B IT: 38.8 (#156), Grok-3 mini: 40.4 (#126)

BenchmarkGemma 4 26B A4B ITGrok-3 mini
WeirdML35.2%42.6%
LMArena Coding14461380
Aider Polyglot—49.3%
LMArena WebDev1359—
SciCode40%—
ALE-Bench927.17—

Reasoning Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 21.1 (#205), Grok-3 mini: 13.8 (#323)

BenchmarkGemma 4 26B A4B ITGrok-3 mini
LMArena Hard Prompts14381374
Epoch Capabilities Index141.85140.35
ARC-AGI-2—0.4%
Kagi LLM Benchmark—61.3%
ARC-AGI-1—16.5%
CritPt0%—
Chess Puzzles6%—
DTBench74.9%—
LMCA29.7%—

Math Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 45.7 (#61), Grok-3 mini: 40.3 (#78)

BenchmarkGemma 4 26B A4B ITGrok-3 mini
OTIS Mock AIME 2024-202582.2%77.8%
LMArena Math14691386
Omni-MATH—31.8%
MATH Level 5—90.9%
FrontierMath (Feb 2025 set)—5.9%

Knowledge Too close to call

Gemma 4 26B A4B IT: 45.6 (#79), Grok-3 mini: 46.3 (#75)

BenchmarkGemma 4 26B A4B ITGrok-3 mini
GPQA Diamond73.2%76.3%
LMArena Expert14471395
MMLU-Pro—79.9%
Confabulations—10.8%
Vectara Hallucination Rate5.2%—
GPQA (HELM)—67.5%

Multimodal Not comparable

Gemma 4 26B A4B IT: 39.9 (#43), Grok-3 mini: —

BenchmarkGemma 4 26B A4B ITGrok-3 mini
LMArena Vision1260—

Multilingual Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 53.3 (#68), Grok-3 mini: 48.3 (#141)

BenchmarkGemma 4 26B A4B ITGrok-3 mini
LMArena Non-English14221352
LMArena Chinese14951387
LMArena French14591357
LMArena Russian14341352
LMArena Spanish14151379
LMArena German—1350
LMArena Japanese—1343
LMArena Korean—1336

Instruction Following Grok-3 mini leads

Gemma 4 26B A4B IT: 75.0 (#78), Grok-3 mini: 78.7 (#8)

BenchmarkGemma 4 26B A4B ITGrok-3 mini
LMArena Instruction Following14201356
IFEval—95.1%

Long Context Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 43.6 (#87), Grok-3 mini: 41.0 (#142)

BenchmarkGemma 4 26B A4B ITGrok-3 mini
LMArena Longer Query14271372
Fiction.LiveBench—66.7%

Writing & Preference Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 58.8 (#110), Grok-3 mini: 52.7 (#163)

BenchmarkGemma 4 26B A4B ITGrok-3 mini
LMArena Text14341370
LMArena Creative Writing14011342
LMArena Multi-Turn14421355
Short-Story Creative Writing—73.5%
EQ-Bench Creative Writing1305—
WildBench—65.1%

Frequently asked questions

Is Gemma 4 26B A4B IT better than Grok-3 mini?

Gemma 4 26B A4B IT is the stronger model overall, scoring 43.1 to 41.0 on the Noometry Index.

Is Gemma 4 26B A4B IT or Grok-3 mini better for coding?

Grok-3 mini scores higher on coding benchmarks: 40.4 versus 38.8 in the Noometry coding category.

How many benchmarks do Gemma 4 26B A4B IT and Grok-3 mini share?

18 benchmarks have published results for both models. Gemma 4 26B A4B IT has 28 scored results on Noometry and Grok-3 mini has 35.

Related comparisons

Go deeper