Noometry

Model comparison

Grok 4 vs o3-mini

Grok 4 is the stronger model overall, scoring 48.1 to 36.7 on the Noometry Index.

Last verified . 33 shared benchmarks.

Grok 4 xAI

48.1

Rank #56 Confirmed

o3-mini OpenAI

36.7

Rank #212 Confirmed

Summary

  • They share 33 benchmarks with published results for both. Grok 4 scores higher in 9 categories and o3-mini in 0 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in long context, where Grok 4 leads 63.1 to 33.8.
  • The biggest single-benchmark swing is Fiction.LiveBench: 94.4% for Grok 4 and 50% for o3-mini.

Side by side

Grok 4o3-mini
ProviderxAIOpenAI
Noometry Index48.136.7
Released2025-07-092024-12-20
WeightsProprietaryProprietary
Context window—200K
Max output—100K
Input $ / M tokens—$1.10
Output $ / M tokens—$4.40
Results tracked4851

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok 4 leads

Grok 4: 50.3 (#46), o3-mini: 40.8 (#132)

BenchmarkGrok 4o3-mini
Aider Polyglot79.6%60.4%
WeirdML45.7%43.7%
LMArena Coding14081378
SciCode—39.8%
GSO—1.3%
LiveBench Coding—82.7%
CadEval—54%

Agentic & Tool Use Grok 4 leads

Grok 4: 32.3 (#68), o3-mini: 29.6 (#84)

BenchmarkGrok 4o3-mini
Cybench43%22.5%
Terminal-Bench27.2%—
Berkeley Function Calling Leaderboard63%—
GDPval21.1%—
DeepResearch Bench47.3%—
BALROG43.6%—
LMArena Search1142—
METR Time Horizons66.6%—

Reasoning Grok 4 leads

Grok 4: 36.7 (#65), o3-mini: 16.3 (#305)

BenchmarkGrok 4o3-mini
ARC-AGI-216%3%
SimpleBench60.5%22.8%
ARC-AGI-166.7%34.5%
Chess Puzzles28%17%
LMArena Hard Prompts14091366
Epoch Capabilities Index146.44140.34
ForecastBench60.959.6
Kagi LLM Benchmark73.6%—
CritPt—0.3%
LiveBench Reasoning—89.6%
Mystery Game Puzzles—7%
DTBench—68.8%
LiveBench Data Analysis—70.6%
LMCA—19%
LiveBench—75.9%

Math Grok 4 leads

Grok 4: 48.4 (#64), o3-mini: 28.1 (#244)

BenchmarkGrok 4o3-mini
OTIS Mock AIME 2024-202584%76.9%
LMArena Math14221396
FrontierMath (Feb 2025 set)19.7%12.4%
FrontierMath Tier 4 (v1)2.1%4.2%
FrontierMath (Tiers 1-3)—18.6%
FrontierMath Tier 4—0%
Omni-MATH60.3%—
LiveBench Math—77.3%
MATH Level 5—96.5%

Knowledge Grok 4 leads

Grok 4: 53.8 (#55), o3-mini: 38.3 (#146)

BenchmarkGrok 4o3-mini
GPQA Diamond87%77%
Confabulations12.4%17.9%
LMArena Expert14151364
SimpleQA Verified—15.3%
MMLU-Pro85.1%—
GPQA (HELM)72.7%—

Multimodal Not comparable

Grok 4: 33.7 (#94), o3-mini: —

BenchmarkGrok 4o3-mini
LMArena Vision1210—
GeoBench45%—

Multilingual Grok 4 leads

Grok 4: 51.8 (#103), o3-mini: 45.7 (#164)

BenchmarkGrok 4o3-mini
LMArena Non-English14031319
LMArena Chinese14271379
LMArena French14181334
LMArena German14291303
LMArena Japanese13941286
LMArena Korean13771314
LMArena Russian14101304
LMArena Spanish14201321

Instruction Following Grok 4 leads

Grok 4: 79.2 (#5), o3-mini: 75.1 (#72)

BenchmarkGrok 4o3-mini
LMArena Instruction Following13871337
LiveBench Instruction Following—84.4%
IFEval94.9%—

Long Context Grok 4 leads

Grok 4: 63.1 (#4), o3-mini: 33.8 (#256)

BenchmarkGrok 4o3-mini
Fiction.LiveBench94.4%50%
LMArena Longer Query14091343

Writing & Preference Grok 4 leads

Grok 4: 58.5 (#116), o3-mini: 50.3 (#182)

BenchmarkGrok 4o3-mini
LMArena Text14111337
LMArena Creative Writing13971286
Short-Story Creative Writing76.9%61.7%
LMArena Multi-Turn14161320
WildBench79.7%—
LiveBench Language—50.7%

Frequently asked questions

Is Grok 4 better than o3-mini?

Grok 4 is the stronger model overall, scoring 48.1 to 36.7 on the Noometry Index.

Is Grok 4 or o3-mini better for coding?

Grok 4 scores higher on coding benchmarks: 50.3 versus 40.8 in the Noometry coding category.

How many benchmarks do Grok 4 and o3-mini share?

33 benchmarks have published results for both models. Grok 4 has 48 scored results on Noometry and o3-mini has 51.

Related comparisons

Go deeper