Noometry

Model comparison

Grok 3 vs o3

o3 is the stronger model overall, scoring 47.5 to 39.9 on the Noometry Index.

Last verified . 38 shared benchmarks.

Grok 3 xAI

39.9

Rank #157 Confirmed

o3 OpenAI

47.5

Rank #61 Confirmed

Summary

  • They share 38 benchmarks with published results for both. Grok 3 scores higher in 2 categories and o3 in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where o3 leads 32.0 to 13.7.
  • The biggest single-benchmark swing is ARC-AGI-1: 5.5% for Grok 3 and 60.8% for o3.

Side by side

Grok 3o3
ProviderxAIOpenAI
Noometry Index39.947.5
Released2025-04-092025-04-16
WeightsProprietaryProprietary
Context window—200K
Max output—100K
Input $ / M tokens—$2
Output $ / M tokens—$8
Results tracked4063

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3 leads

Grok 3: 41.9 (#115), o3: 46.8 (#64)

BenchmarkGrok 3o3
Aider Polyglot53.3%81.3%
WeirdML37.2%52.4%
LMArena Coding14321408
SWE-bench Verified—62.3%
SWE-bench Verified (bash only)—58.4%
GSO—8.8%
CadEval—74%
ALE-Bench—933.55

Agentic & Tool Use o3 leads

Grok 3: 30.5 (#76), o3: 34.5 (#44)

BenchmarkGrok 3o3
Berkeley Function Calling Leaderboard—63%
GDPval—30.8%
DeepResearch Bench—45.2%
OSWorld—23%
BALROG29.5%—
LMArena Search—1144
METR Time Horizons—65.4%

Reasoning o3 leads

Grok 3: 13.7 (#333), o3: 32.0 (#78)

BenchmarkGrok 3o3
ARC-AGI-20%6.5%
SimpleBench36.1%53.1%
Kagi LLM Benchmark61.3%67.6%
ARC-AGI-15.5%60.8%
LMArena Hard Prompts14341402
Epoch Capabilities Index138.33146.86
CritPt—1.4%
Chess Puzzles—38%
EnigmaEval—13.1%
Mystery Game Puzzles—29%
DTBench—84.8%
LMCA—39.7%
ForecastBench—62.5

Math o3 leads

Grok 3: 38.0 (#145), o3: 50.2 (#58)

BenchmarkGrok 3o3
OTIS Mock AIME 2024-202555.6%84.4%
Omni-MATH46.4%71.4%
LMArena Math13911426
MATH Level 588.7%97.8%
FrontierMath (Feb 2025 set)3.8%18.7%
FrontierMath Tier 4 (v1)0%2.1%
FrontierMath (Tiers 1-3)—33.3%

Knowledge o3 leads

Grok 3: 46.2 (#82), o3: 54.6 (#52)

BenchmarkGrok 3o3
GPQA Diamond75.8%81.8%
MMLU-Pro78.8%85.9%
Confabulations14.2%14.4%
GPQA (HELM)65%75.3%
LMArena Expert14211402
Humanity's Last Exam—20.3%
SimpleQA Verified—49.4%
Vectara Hallucination Rate5.8%—

Multimodal Not comparable

Grok 3: —, o3: 41.4 (#36)

BenchmarkGrok 3o3
LMArena Vision—1214
GeoBench—74%
VPCT—52%

Multilingual Too close to call

Grok 3: 52.3 (#87), o3: 51.7 (#105)

BenchmarkGrok 3o3
LMArena Non-English14101401
LMArena Chinese14481437
LMArena French14601430
LMArena German14311420
LMArena Japanese13871403
LMArena Korean13731370
LMArena Russian14161406
LMArena Spanish14171395

Instruction Following Grok 3 leads

Grok 3: 75.0 (#73), o3: 72.8 (#127)

BenchmarkGrok 3o3
IFEval88.4%86.9%
LMArena Instruction Following14091368

Long Context o3 leads

Grok 3: 38.7 (#192), o3: 53.3 (#6)

BenchmarkGrok 3o3
Fiction.LiveBench58.3%88.9%
LMArena Longer Query14391372
CL-bench—17.8%

Writing & Preference o3 leads

Grok 3: 55.8 (#141), o3: 63.5 (#64)

BenchmarkGrok 3o3
LMArena Text14261410
LMArena Creative Writing14141359
Short-Story Creative Writing76.4%83.9%
EQ-Bench Creative Writing11861676
WildBench84.9%86.1%
LMArena Multi-Turn14251405

Frequently asked questions

Is Grok 3 better than o3?

o3 is the stronger model overall, scoring 47.5 to 39.9 on the Noometry Index.

Is Grok 3 or o3 better for coding?

o3 scores higher on coding benchmarks: 46.8 versus 41.9 in the Noometry coding category.

How many benchmarks do Grok 3 and o3 share?

38 benchmarks have published results for both models. Grok 3 has 40 scored results on Noometry and o3 has 63.

Related comparisons

Go deeper