Noometry

Model comparison

GPT-4o mini vs Grok-3 mini

Grok-3 mini is the stronger model overall, scoring 41.2 to 25.5 on the Noometry Index.

Last verified . 32 shared benchmarks.

GPT-4o mini OpenAI

25.5

Rank #343 Confirmed

Grok-3 mini xAI

41.2

Rank #141 Confirmed

Summary

  • They share 32 benchmarks with published results for both. GPT-4o mini scores higher in 0 categories and Grok-3 mini in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Grok-3 mini leads 42.1 to 10.4.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 6.9% for GPT-4o mini and 77.8% for Grok-3 mini.

Side by side

GPT-4o miniGrok-3 mini
ProviderOpenAIxAI
Noometry Index25.541.2
Released2024-07-182025-04-09
WeightsProprietaryProprietary
Context window128K—
Max output16K—
Input $ / M tokens$0.15—
Output $ / M tokens$0.60—
Results tracked6035

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok-3 mini leads

GPT-4o mini: 22.0 (#335), Grok-3 mini: 40.8 (#131)

BenchmarkGPT-4o miniGrok-3 mini
Aider Polyglot3.6%49.3%
WeirdML11.8%42.6%
LMArena Coding12901379
BigCodeBench Instruct46.1%—
LiveBench Coding43.1%—
BigCodeBench Complete57.4%—
HumanEval+83.5%—
MBPP+72.2%—

Agentic & Tool Use Not comparable

GPT-4o mini: 27.5 (#101), Grok-3 mini: —

BenchmarkGPT-4o miniGrok-3 mini
BALROG17.4%—

Reasoning Grok-3 mini leads

GPT-4o mini: 8.7 (#347), Grok-3 mini: 13.6 (#334)

BenchmarkGPT-4o miniGrok-3 mini
ARC-AGI-20%0.4%
Kagi LLM Benchmark28.8%61.3%
LMArena Hard Prompts12671375
Epoch Capabilities Index126.56140.35
SimpleBench10.7%—
ARC-AGI-1—16.5%
Chess Puzzles0%—
LiveBench Reasoning32.8%—
Mystery Game Puzzles12%—
DTBench54.4%—
LiveBench Data Analysis50%—
LMCA10.4%—
LiveBench41.3%—
PIQA88.7%—

Math Grok-3 mini leads

GPT-4o mini: 10.4 (#314), Grok-3 mini: 42.1 (#85)

BenchmarkGPT-4o miniGrok-3 mini
OTIS Mock AIME 2024-20256.9%77.8%
Omni-MATH28%31.8%
LMArena Math12671386
MATH Level 552.6%90.9%
FrontierMath (Tiers 1-3)0.7%—
LiveBench Math36.3%—
FrontierMath (Feb 2025 set)—5.9%
GSM8K91.3%—

Knowledge Grok-3 mini leads

GPT-4o mini: 17.7 (#284), Grok-3 mini: 46.4 (#81)

BenchmarkGPT-4o miniGrok-3 mini
GPQA Diamond37.7%76.3%
MMLU-Pro60.3%79.9%
Confabulations37.2%10.8%
GPQA (HELM)36.8%67.5%
LMArena Expert12351395
SimpleQA Verified8.3%—
BoolQ88.7%—
MMLU81.8%—

Multimodal Not comparable

GPT-4o mini: 25.9 (#122), Grok-3 mini: —

BenchmarkGPT-4o miniGrok-3 mini
LMArena Vision1066—
Video-MME64.8%—
GeoBench64%—
VPCT34%—

Multilingual Grok-3 mini leads

GPT-4o mini: 42.0 (#199), Grok-3 mini: 48.1 (#145)

BenchmarkGPT-4o miniGrok-3 mini
LMArena Non-English12661352
LMArena Chinese12651387
LMArena French12971357
LMArena German12721349
LMArena Japanese12161342
LMArena Korean11951335
LMArena Russian12751353
LMArena Spanish12761381

Instruction Following Grok-3 mini leads

GPT-4o mini: 61.9 (#239), Grok-3 mini: 78.5 (#9)

BenchmarkGPT-4o miniGrok-3 mini
IFEval78.2%95.1%
LMArena Instruction Following12581357
LiveBench Instruction Following56.8%—

Long Context Grok-3 mini leads

GPT-4o mini: 39.1 (#186), Grok-3 mini: 41.0 (#147)

BenchmarkGPT-4o miniGrok-3 mini
LMArena Longer Query12891372
Fiction.LiveBench—66.7%

Writing & Preference Grok-3 mini leads

GPT-4o mini: 39.5 (#248), Grok-3 mini: 52.5 (#169)

BenchmarkGPT-4o miniGrok-3 mini
LMArena Text12861370
LMArena Creative Writing12681342
Short-Story Creative Writing67.2%73.5%
WildBench79.1%65.1%
LMArena Multi-Turn12851355
EQ-Bench Creative Writing873—
LiveBench Language28.6%—

Frequently asked questions

Is GPT-4o mini better than Grok-3 mini?

Grok-3 mini is the stronger model overall, scoring 41.2 to 25.5 on the Noometry Index.

Is GPT-4o mini or Grok-3 mini better for coding?

Grok-3 mini scores higher on coding benchmarks: 40.8 versus 22.0 in the Noometry coding category.

How many benchmarks do GPT-4o mini and Grok-3 mini share?

32 benchmarks have published results for both models. GPT-4o mini has 60 scored results on Noometry and Grok-3 mini has 35.

Related comparisons

Go deeper