Noometry

Model comparison

Gemini 1.5 Flash 8B vs Grok-3 mini

Grok-3 mini is the stronger model overall, scoring 41.0 to 29.7 on the Noometry Index.

Last verified . 19 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.7

Rank #292 Confirmed

Grok-3 mini xAI

41.0

Rank #130 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Gemini 1.5 Flash 8B scores higher in 1 category and Grok-3 mini in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Grok-3 mini leads 46.3 to 16.0.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 4.6% for Gemini 1.5 Flash 8B and 77.8% for Grok-3 mini.

Side by side

Gemini 1.5 Flash 8BGrok-3 mini
ProviderGooglexAI
Noometry Index29.741.0
Released2024-10-032025-04-09
WeightsProprietaryProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked2135

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok-3 mini leads

Gemini 1.5 Flash 8B: 34.9 (#221), Grok-3 mini: 40.4 (#126)

BenchmarkGemini 1.5 Flash 8BGrok-3 mini
LMArena Coding12181380
Aider Polyglot—49.3%
WeirdML—42.6%

Reasoning Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 19.2 (#242), Grok-3 mini: 13.8 (#323)

BenchmarkGemini 1.5 Flash 8BGrok-3 mini
LMArena Hard Prompts12101374
ARC-AGI-2—0.4%
Kagi LLM Benchmark—61.3%
ARC-AGI-1—16.5%
DTBench50%—
Epoch Capabilities Index—140.35

Math Grok-3 mini leads

Gemini 1.5 Flash 8B: 13.3 (#296), Grok-3 mini: 40.3 (#78)

BenchmarkGemini 1.5 Flash 8BGrok-3 mini
OTIS Mock AIME 2024-20254.6%77.8%
LMArena Math12071386
Omni-MATH—31.8%
MATH Level 5—90.9%
FrontierMath (Feb 2025 set)—5.9%

Knowledge Grok-3 mini leads

Gemini 1.5 Flash 8B: 16.0 (#285), Grok-3 mini: 46.3 (#75)

BenchmarkGemini 1.5 Flash 8BGrok-3 mini
GPQA Diamond33%76.3%
LMArena Expert11851395
MMLU-Pro—79.9%
Confabulations—10.8%
GPQA (HELM)—67.5%

Multimodal Not comparable

Gemini 1.5 Flash 8B: 27.6 (#110), Grok-3 mini: —

BenchmarkGemini 1.5 Flash 8BGrok-3 mini
LMArena Vision1044—

Multilingual Grok-3 mini leads

Gemini 1.5 Flash 8B: 38.6 (#224), Grok-3 mini: 48.3 (#141)

BenchmarkGemini 1.5 Flash 8BGrok-3 mini
LMArena Non-English12151352
LMArena Chinese12311387
LMArena French12331357
LMArena German12061350
LMArena Japanese11501343
LMArena Korean11401336
LMArena Russian12361352
LMArena Spanish12121379

Instruction Following Grok-3 mini leads

Gemini 1.5 Flash 8B: 63.1 (#231), Grok-3 mini: 78.7 (#8)

BenchmarkGemini 1.5 Flash 8BGrok-3 mini
LMArena Instruction Following11991356
IFEval—95.1%

Long Context Grok-3 mini leads

Gemini 1.5 Flash 8B: 36.9 (#220), Grok-3 mini: 41.0 (#142)

BenchmarkGemini 1.5 Flash 8BGrok-3 mini
LMArena Longer Query12191372
Fiction.LiveBench—66.7%

Writing & Preference Grok-3 mini leads

Gemini 1.5 Flash 8B: 43.0 (#227), Grok-3 mini: 52.7 (#163)

BenchmarkGemini 1.5 Flash 8BGrok-3 mini
LMArena Text12261370
LMArena Creative Writing12181342
LMArena Multi-Turn11851355
Short-Story Creative Writing—73.5%
WildBench—65.1%

Frequently asked questions

Is Gemini 1.5 Flash 8B better than Grok-3 mini?

Grok-3 mini is the stronger model overall, scoring 41.0 to 29.7 on the Noometry Index.

Is Gemini 1.5 Flash 8B or Grok-3 mini better for coding?

Grok-3 mini scores higher on coding benchmarks: 40.4 versus 34.9 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash 8B and Grok-3 mini share?

19 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and Grok-3 mini has 35.

Related comparisons

Go deeper