Noometry

Model comparison

Grok 4 Fast vs o1

o1 is the stronger model overall, scoring 40.9 to 39.4 on the Noometry Index.

Last verified . 23 shared benchmarks.

Grok 4 Fast xAI

39.4

Rank #167 Confirmed

o1 OpenAI

40.9

Rank #143 Confirmed

Summary

  • They share 23 benchmarks with published results for both. Grok 4 Fast scores higher in 5 categories and o1 in 4 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in coding, where o1 leads 46.1 to 32.5.
  • The biggest single-benchmark swing is Cybench: 30% for Grok 4 Fast and 10% for o1.

Side by side

Grok 4 Fasto1
ProviderxAIOpenAI
Noometry Index39.440.9
Released2025-09-192024-09-12
WeightsProprietaryProprietary
Context window—200K
Max output—100K
Input $ / M tokens—$15
Output $ / M tokens—$60
Results tracked3052

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o1 leads

Grok 4 Fast: 32.5 (#271), o1: 46.1 (#70)

BenchmarkGrok 4 Fasto1
WeirdML42.9%47.6%
LMArena Coding14291367
Aider Polyglot—61.7%
LMArena WebDev1159—
LiveBench Coding—69.7%
CadEval—56%
HumanEval+—89%
MBPP+—80.2%

Agentic & Tool Use Grok 4 Fast leads

Grok 4 Fast: 29.5 (#86), o1: 24.6 (#117)

BenchmarkGrok 4 Fasto1
Cybench30%10%
τ²-bench Banking15.7%—
LMArena Search1171—
METR Time Horizons—51.1%

Reasoning o1 leads

Grok 4 Fast: 22.2 (#201), o1: 27.9 (#111)

BenchmarkGrok 4 Fasto1
ARC-AGI-148.5%30.7%
LMArena Hard Prompts14121371
DTBench82.7%74.7%
Epoch Capabilities Index144.2141.91
ARC-AGI-25.3%—
SimpleBench—41.7%
Kagi LLM Benchmark66.1%—
Chess Puzzles—15%
EnigmaEval—5.7%
LiveBench Reasoning—91.6%
LiveBench Data Analysis—65.5%
LMCA—22.3%
ForecastBench60.5—
LiveBench—75.7%

Math Grok 4 Fast leads

Grok 4 Fast: 38.9 (#123), o1: 36.1 (#175)

BenchmarkGrok 4 Fasto1
LMArena Math14191388
FrontierMath (Tiers 1-3)—14.7%
OTIS Mock AIME 2024-2025—73.3%
LiveBench Math—80.3%
MATH Level 5—94.7%
FrontierMath (Feb 2025 set)—9.3%

Knowledge o1 leads

Grok 4 Fast: 32.0 (#214), o1: 41.5 (#110)

BenchmarkGrok 4 Fasto1
LMArena Expert14111361
GPQA Diamond—76.8%
Humanity's Last Exam—8%
SimpleQA Verified—41.1%
Confabulations—11.7%
Vectara Hallucination Rate19.7%—

Multimodal Not comparable

Grok 4 Fast: —, o1: 34.2 (#93)

BenchmarkGrok 4 Fasto1
LMArena Vision—1168
GeoBench—80%
VPCT—37%
SpatialViz-Bench—41.4%

Multilingual Grok 4 Fast leads

Grok 4 Fast: 51.3 (#111), o1: 48.6 (#142)

BenchmarkGrok 4 Fasto1
LMArena Non-English13961358
LMArena Chinese14571394
LMArena French14311344
LMArena German13831337
LMArena Japanese13521346
LMArena Korean13571396
LMArena Russian13891356
LMArena Spanish14151345

Instruction Following o1 leads

Grok 4 Fast: 73.2 (#121), o1: 74.8 (#86)

BenchmarkGrok 4 Fasto1
LMArena Instruction Following13871367
LiveBench Instruction Following—81.5%

Long Context Grok 4 Fast leads

Grok 4 Fast: 63.2 (#3), o1: 50.3 (#9)

BenchmarkGrok 4 Fasto1
Fiction.LiveBench94.4%83.3%
LMArena Longer Query14151378

Writing & Preference Grok 4 Fast leads

Grok 4 Fast: 60.0 (#102), o1: 55.6 (#144)

BenchmarkGrok 4 Fasto1
LMArena Text14071366
LMArena Creative Writing13871348
LMArena Multi-Turn14141369
Short-Story Creative Writing—70.2%
LiveBench Language—65.4%

Frequently asked questions

Is Grok 4 Fast better than o1?

o1 is the stronger model overall, scoring 40.9 to 39.4 on the Noometry Index.

Is Grok 4 Fast or o1 better for coding?

o1 scores higher on coding benchmarks: 46.1 versus 32.5 in the Noometry coding category.

How many benchmarks do Grok 4 Fast and o1 share?

23 benchmarks have published results for both models. Grok 4 Fast has 30 scored results on Noometry and o1 has 52.

Related comparisons

Go deeper