Noometry

Model comparison

Grok 4 Fast vs o3-mini

Grok 4 Fast is the stronger model overall, scoring 39.4 to 36.7 on the Noometry Index.

Last verified . 25 shared benchmarks.

Grok 4 Fast xAI

39.4

Rank #167 Confirmed

o3-mini OpenAI

36.7

Rank #212 Confirmed

Summary

  • They share 25 benchmarks with published results for both. Grok 4 Fast scores higher in 5 categories and o3-mini in 4 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in long context, where Grok 4 Fast leads 63.2 to 33.8.
  • The biggest single-benchmark swing is Fiction.LiveBench: 94.4% for Grok 4 Fast and 50% for o3-mini.

Side by side

Grok 4 Fasto3-mini
ProviderxAIOpenAI
Noometry Index39.436.7
Released2025-09-192024-12-20
WeightsProprietaryProprietary
Context window—200K
Max output—100K
Input $ / M tokens—$1.10
Output $ / M tokens—$4.40
Results tracked3051

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3-mini leads

Grok 4 Fast: 32.5 (#271), o3-mini: 40.8 (#132)

BenchmarkGrok 4 Fasto3-mini
WeirdML42.9%43.7%
LMArena Coding14291378
Aider Polyglot—60.4%
LMArena WebDev1159—
SciCode—39.8%
GSO—1.3%
LiveBench Coding—82.7%
CadEval—54%

Agentic & Tool Use Too close to call

Grok 4 Fast: 29.5 (#86), o3-mini: 29.6 (#84)

BenchmarkGrok 4 Fasto3-mini
Cybench30%22.5%
τ²-bench Banking15.7%—
LMArena Search1171—

Reasoning Grok 4 Fast leads

Grok 4 Fast: 22.2 (#201), o3-mini: 16.3 (#305)

BenchmarkGrok 4 Fasto3-mini
ARC-AGI-25.3%3%
ARC-AGI-148.5%34.5%
LMArena Hard Prompts14121366
DTBench82.7%68.8%
Epoch Capabilities Index144.2140.34
ForecastBench60.559.6
SimpleBench—22.8%
Kagi LLM Benchmark66.1%—
CritPt—0.3%
Chess Puzzles—17%
LiveBench Reasoning—89.6%
Mystery Game Puzzles—7%
LiveBench Data Analysis—70.6%
LMCA—19%
LiveBench—75.9%

Math Grok 4 Fast leads

Grok 4 Fast: 38.9 (#123), o3-mini: 28.1 (#244)

BenchmarkGrok 4 Fasto3-mini
LMArena Math14191396
FrontierMath (Tiers 1-3)—18.6%
FrontierMath Tier 4—0%
OTIS Mock AIME 2024-2025—76.9%
LiveBench Math—77.3%
MATH Level 5—96.5%
FrontierMath (Feb 2025 set)—12.4%
FrontierMath Tier 4 (v1)—4.2%

Knowledge o3-mini leads

Grok 4 Fast: 32.0 (#214), o3-mini: 38.3 (#146)

BenchmarkGrok 4 Fasto3-mini
LMArena Expert14111364
GPQA Diamond—77%
SimpleQA Verified—15.3%
Confabulations—17.9%
Vectara Hallucination Rate19.7%—

Multilingual Grok 4 Fast leads

Grok 4 Fast: 51.3 (#111), o3-mini: 45.7 (#164)

BenchmarkGrok 4 Fasto3-mini
LMArena Non-English13961319
LMArena Chinese14571379
LMArena French14311334
LMArena German13831303
LMArena Japanese13521286
LMArena Korean13571314
LMArena Russian13891304
LMArena Spanish14151321

Instruction Following o3-mini leads

Grok 4 Fast: 73.2 (#121), o3-mini: 75.1 (#72)

BenchmarkGrok 4 Fasto3-mini
LMArena Instruction Following13871337
LiveBench Instruction Following—84.4%

Long Context Grok 4 Fast leads

Grok 4 Fast: 63.2 (#3), o3-mini: 33.8 (#256)

BenchmarkGrok 4 Fasto3-mini
Fiction.LiveBench94.4%50%
LMArena Longer Query14151343

Writing & Preference Grok 4 Fast leads

Grok 4 Fast: 60.0 (#102), o3-mini: 50.3 (#182)

BenchmarkGrok 4 Fasto3-mini
LMArena Text14071337
LMArena Creative Writing13871286
LMArena Multi-Turn14141320
Short-Story Creative Writing—61.7%
LiveBench Language—50.7%

Frequently asked questions

Is Grok 4 Fast better than o3-mini?

Grok 4 Fast is the stronger model overall, scoring 39.4 to 36.7 on the Noometry Index.

Is Grok 4 Fast or o3-mini better for coding?

o3-mini scores higher on coding benchmarks: 40.8 versus 32.5 in the Noometry coding category.

How many benchmarks do Grok 4 Fast and o3-mini share?

25 benchmarks have published results for both models. Grok 4 Fast has 30 scored results on Noometry and o3-mini has 51.

Related comparisons

Go deeper