Noometry

Model comparison

Gemini 1.5 Flash 8B vs Grok 4 Fast

Grok 4 Fast is the stronger model overall, scoring 39.0 to 29.7 on the Noometry Index.

Last verified . 18 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.7

Rank #292 Confirmed

Grok 4 Fast xAI

39.0

Rank #164 Confirmed

Summary

  • They share 18 benchmarks with published results for both. Gemini 1.5 Flash 8B scores higher in 1 category and Grok 4 Fast in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in long context, where Grok 4 Fast leads 63.2 to 36.9.
  • The biggest single-benchmark swing is DTBench: 50% for Gemini 1.5 Flash 8B and 82.7% for Grok 4 Fast.

Side by side

Gemini 1.5 Flash 8BGrok 4 Fast
ProviderGooglexAI
Noometry Index29.739.0
Released2024-10-032025-09-19
WeightsProprietaryProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked2130

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 34.9 (#221), Grok 4 Fast: 32.3 (#264)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Coding12181429
LMArena WebDev—1160
WeirdML—42.9%

Agentic & Tool Use Not comparable

Gemini 1.5 Flash 8B: —, Grok 4 Fast: 29.3 (#80)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
τ²-bench Banking—15.7%
Cybench—30%
LMArena Search—1171

Reasoning Grok 4 Fast leads

Gemini 1.5 Flash 8B: 19.2 (#242), Grok 4 Fast: 22.5 (#179)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Hard Prompts12101413
DTBench50%82.7%
ARC-AGI-2—5.3%
Kagi LLM Benchmark—66.1%
ARC-AGI-1—48.5%
Epoch Capabilities Index—144.21
ForecastBench—60.5

Math Grok 4 Fast leads

Gemini 1.5 Flash 8B: 13.3 (#296), Grok 4 Fast: 36.1 (#126)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Math12071419
OTIS Mock AIME 2024-20254.6%—

Knowledge Grok 4 Fast leads

Gemini 1.5 Flash 8B: 16.0 (#285), Grok 4 Fast: 31.5 (#211)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Expert11851411
GPQA Diamond33%—
Vectara Hallucination Rate—19.7%

Multimodal Not comparable

Gemini 1.5 Flash 8B: 27.6 (#110), Grok 4 Fast: —

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Vision1044—

Multilingual Grok 4 Fast leads

Gemini 1.5 Flash 8B: 38.6 (#224), Grok 4 Fast: 51.5 (#106)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Non-English12151397
LMArena Chinese12311458
LMArena French12331433
LMArena German12061383
LMArena Japanese11501358
LMArena Korean11401359
LMArena Russian12361387
LMArena Spanish12121414

Instruction Following Grok 4 Fast leads

Gemini 1.5 Flash 8B: 63.1 (#231), Grok 4 Fast: 73.4 (#116)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Instruction Following11991388

Long Context Grok 4 Fast leads

Gemini 1.5 Flash 8B: 36.9 (#220), Grok 4 Fast: 63.2 (#3)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Longer Query12191415
Fiction.LiveBench—94.4%

Writing & Preference Grok 4 Fast leads

Gemini 1.5 Flash 8B: 43.0 (#227), Grok 4 Fast: 60.2 (#96)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Text12261407
LMArena Creative Writing12181388
LMArena Multi-Turn11851414

Frequently asked questions

Is Gemini 1.5 Flash 8B better than Grok 4 Fast?

Grok 4 Fast is the stronger model overall, scoring 39.0 to 29.7 on the Noometry Index.

Is Gemini 1.5 Flash 8B or Grok 4 Fast better for coding?

Gemini 1.5 Flash 8B scores higher on coding benchmarks: 34.9 versus 32.3 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash 8B and Grok 4 Fast share?

18 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and Grok 4 Fast has 30.

Related comparisons

Go deeper