Noometry

Model comparison

GPT-5.4 mini vs Grok 3

GPT-5.4 mini is the stronger model overall, scoring 45.0 to 39.9 on the Noometry Index.

Last verified . 28 shared benchmarks.

GPT-5.4 mini OpenAI

45.0

Rank #76 Confirmed

Grok 3 xAI

39.9

Rank #157 Confirmed

Summary

  • They share 28 benchmarks with published results for both. GPT-5.4 mini scores higher in 6 categories and Grok 3 in 3 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where GPT-5.4 mini leads 30.4 to 13.7.
  • The biggest single-benchmark swing is ARC-AGI-1: 63.7% for GPT-5.4 mini and 5.5% for Grok 3.

Side by side

GPT-5.4 miniGrok 3
ProviderOpenAIxAI
Noometry Index45.039.9
Released2026-03-172025-04-09
WeightsProprietaryProprietary
Context window400K—
Max output128K—
Input $ / M tokens$0.75—
Output $ / M tokens$4.50—
Results tracked4640

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.4 mini leads

GPT-5.4 mini: 45.2 (#72), Grok 3: 41.9 (#115)

BenchmarkGPT-5.4 miniGrok 3
WeirdML60.3%37.2%
LMArena Coding14381432
FrontierCode27%—
Aider Polyglot—53.3%
LMArena WebDev1397—
SciCode49.9%—
ALE-Bench1,189—

Agentic & Tool Use Too close to call

GPT-5.4 mini: 29.9 (#81), Grok 3: 30.5 (#76)

BenchmarkGPT-5.4 miniGrok 3
DeepResearch Bench36.3%—
BALROG—29.5%

Reasoning GPT-5.4 mini leads

GPT-5.4 mini: 30.4 (#85), Grok 3: 13.7 (#333)

BenchmarkGPT-5.4 miniGrok 3
ARC-AGI-218.9%0%
Kagi LLM Benchmark37.9%61.3%
ARC-AGI-163.7%5.5%
LMArena Hard Prompts14241434
Epoch Capabilities Index148.84138.33
SimpleBench—36.1%
NYT Connections (extended)61.8%—
CritPt10%—
Chess Puzzles24%—
Thematic Generalization61.7%—
Mystery Game Puzzles11%—
DTBench80%—
LMCA40.8%—
ForecastBench57—

Math GPT-5.4 mini leads

GPT-5.4 mini: 45.5 (#75), Grok 3: 38.0 (#145)

BenchmarkGPT-5.4 miniGrok 3
OTIS Mock AIME 2024-202588.9%55.6%
LMArena Math14191391
FrontierMath (Feb 2025 set)28.3%3.8%
FrontierMath Tier 4 (v1)2.1%0%
FrontierMath (Tiers 1-3)51.2%—
FrontierMath Tier 49.8%—
ProofBench21%—
Omni-MATH—46.4%
MATH Level 5—88.7%

Knowledge GPT-5.4 mini leads

GPT-5.4 mini: 51.5 (#67), Grok 3: 46.2 (#82)

BenchmarkGPT-5.4 miniGrok 3
GPQA Diamond86.9%75.8%
Vectara Hallucination Rate5.5%5.8%
LMArena Expert14351421
SimpleQA Verified29.4%—
MMLU-Pro—78.8%
Confabulations—14.2%
GPQA (HELM)—65%

Multimodal Not comparable

GPT-5.4 mini: 39.7 (#56), Grok 3: —

BenchmarkGPT-5.4 miniGrok 3
LMArena Vision1245—

Multilingual Too close to call

GPT-5.4 mini: 51.9 (#96), Grok 3: 52.3 (#87)

BenchmarkGPT-5.4 miniGrok 3
LMArena Non-English14051410
LMArena Chinese14461448
LMArena French14401460
LMArena German14091431
LMArena Japanese13741387
LMArena Korean13681373
LMArena Russian14171416
LMArena Spanish14051417

Instruction Following Too close to call

GPT-5.4 mini: 74.1 (#102), Grok 3: 75.0 (#73)

BenchmarkGPT-5.4 miniGrok 3
LMArena Instruction Following14051409
IFEval—88.4%

Long Context GPT-5.4 mini leads

GPT-5.4 mini: 43.0 (#112), Grok 3: 38.7 (#192)

BenchmarkGPT-5.4 miniGrok 3
LMArena Longer Query14071439
Fiction.LiveBench—58.3%

Writing & Preference GPT-5.4 mini leads

GPT-5.4 mini: 64.0 (#58), Grok 3: 55.8 (#141)

BenchmarkGPT-5.4 miniGrok 3
LMArena Text14121426
LMArena Creative Writing13701414
EQ-Bench Creative Writing16651186
LMArena Multi-Turn14291425
Short-Story Creative Writing—76.4%
WildBench—84.9%

Frequently asked questions

Is GPT-5.4 mini better than Grok 3?

GPT-5.4 mini is the stronger model overall, scoring 45.0 to 39.9 on the Noometry Index.

Is GPT-5.4 mini or Grok 3 better for coding?

GPT-5.4 mini scores higher on coding benchmarks: 45.2 versus 41.9 in the Noometry coding category.

How many benchmarks do GPT-5.4 mini and Grok 3 share?

28 benchmarks have published results for both models. GPT-5.4 mini has 46 scored results on Noometry and Grok 3 has 40.

Related comparisons

Go deeper