Noometry

Model comparison

Gemini 1.5 Flash 8B vs Grok 3

Grok 3 is the stronger model overall, scoring 39.9 to 29.9 on the Noometry Index.

Last verified . 19 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.9

Rank #301 Confirmed

Grok 3 xAI

39.9

Rank #157 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Gemini 1.5 Flash 8B scores higher in 1 category and Grok 3 in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Grok 3 leads 46.2 to 16.0.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 4.6% for Gemini 1.5 Flash 8B and 55.6% for Grok 3.

Side by side

Gemini 1.5 Flash 8BGrok 3
ProviderGooglexAI
Noometry Index29.939.9
Released2024-10-032025-04-09
WeightsProprietaryProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked2140

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok 3 leads

Gemini 1.5 Flash 8B: 35.5 (#225), Grok 3: 41.9 (#115)

BenchmarkGemini 1.5 Flash 8BGrok 3
LMArena Coding12181432
Aider Polyglot—53.3%
WeirdML—37.2%

Agentic & Tool Use Not comparable

Gemini 1.5 Flash 8B: —, Grok 3: 30.5 (#76)

BenchmarkGemini 1.5 Flash 8BGrok 3
BALROG—29.5%

Reasoning Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 20.0 (#244), Grok 3: 13.7 (#333)

BenchmarkGemini 1.5 Flash 8BGrok 3
LMArena Hard Prompts12091434
ARC-AGI-2—0%
SimpleBench—36.1%
Kagi LLM Benchmark—61.3%
ARC-AGI-1—5.5%
DTBench50%—
Epoch Capabilities Index—138.33

Math Grok 3 leads

Gemini 1.5 Flash 8B: 14.2 (#302), Grok 3: 38.0 (#145)

BenchmarkGemini 1.5 Flash 8BGrok 3
OTIS Mock AIME 2024-20254.6%55.6%
LMArena Math12071391
Omni-MATH—46.4%
MATH Level 5—88.7%
FrontierMath (Feb 2025 set)—3.8%
FrontierMath Tier 4 (v1)—0%

Knowledge Grok 3 leads

Gemini 1.5 Flash 8B: 16.0 (#289), Grok 3: 46.2 (#82)

BenchmarkGemini 1.5 Flash 8BGrok 3
GPQA Diamond33%75.8%
LMArena Expert11851421
MMLU-Pro—78.8%
Confabulations—14.2%
Vectara Hallucination Rate—5.8%
GPQA (HELM)—65%

Multimodal Not comparable

Gemini 1.5 Flash 8B: 28.2 (#115), Grok 3: —

BenchmarkGemini 1.5 Flash 8BGrok 3
LMArena Vision1044—

Multilingual Grok 3 leads

Gemini 1.5 Flash 8B: 38.5 (#229), Grok 3: 52.3 (#87)

BenchmarkGemini 1.5 Flash 8BGrok 3
LMArena Non-English12151410
LMArena Chinese12311448
LMArena French12341460
LMArena German12061431
LMArena Japanese11501387
LMArena Korean11401373
LMArena Russian12361416
LMArena Spanish12121417

Instruction Following Grok 3 leads

Gemini 1.5 Flash 8B: 62.8 (#236), Grok 3: 75.0 (#73)

BenchmarkGemini 1.5 Flash 8BGrok 3
LMArena Instruction Following11991409
IFEval—88.4%

Long Context Grok 3 leads

Gemini 1.5 Flash 8B: 37.0 (#225), Grok 3: 38.7 (#192)

BenchmarkGemini 1.5 Flash 8BGrok 3
LMArena Longer Query12191439
Fiction.LiveBench—58.3%

Writing & Preference Grok 3 leads

Gemini 1.5 Flash 8B: 42.8 (#232), Grok 3: 55.8 (#141)

BenchmarkGemini 1.5 Flash 8BGrok 3
LMArena Text12261426
LMArena Creative Writing12181414
LMArena Multi-Turn11851425
Short-Story Creative Writing—76.4%
EQ-Bench Creative Writing—1186
WildBench—84.9%

Frequently asked questions

Is Gemini 1.5 Flash 8B better than Grok 3?

Grok 3 is the stronger model overall, scoring 39.9 to 29.9 on the Noometry Index.

Is Gemini 1.5 Flash 8B or Grok 3 better for coding?

Grok 3 scores higher on coding benchmarks: 41.9 versus 35.5 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash 8B and Grok 3 share?

19 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and Grok 3 has 40.

Related comparisons

Go deeper