Noometry

Model comparison

GPT-5.4 nano vs Grok 3

GPT-5.4 nano is the stronger model overall, scoring 41.9 to 39.9 on the Noometry Index.

Last verified . 27 shared benchmarks.

GPT-5.4 nano OpenAI

41.9

Rank #125 Confirmed

Grok 3 xAI

39.9

Rank #157 Confirmed

Summary

  • They share 27 benchmarks with published results for both. GPT-5.4 nano scores higher in 4 categories and Grok 3 in 4 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where GPT-5.4 nano leads 23.7 to 13.7.
  • The biggest single-benchmark swing is ARC-AGI-1: 51.5% for GPT-5.4 nano and 5.5% for Grok 3.

Side by side

GPT-5.4 nanoGrok 3
ProviderOpenAIxAI
Noometry Index41.939.9
Released2026-03-172025-04-09
WeightsProprietaryProprietary
Context window400K—
Max output128K—
Input $ / M tokens$0.20—
Output $ / M tokens$1.25—
Results tracked4040

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.4 nano leads

GPT-5.4 nano: 43.6 (#84), Grok 3: 41.9 (#115)

BenchmarkGPT-5.4 nanoGrok 3
WeirdML49.2%37.2%
LMArena Coding14051432
Aider Polyglot—53.3%
SciCode46.9%—
ALE-Bench1,005—

Agentic & Tool Use Not comparable

GPT-5.4 nano: —, Grok 3: 30.5 (#76)

BenchmarkGPT-5.4 nanoGrok 3
BALROG—29.5%

Reasoning GPT-5.4 nano leads

GPT-5.4 nano: 23.7 (#173), Grok 3: 13.7 (#333)

BenchmarkGPT-5.4 nanoGrok 3
ARC-AGI-25.7%0%
Kagi LLM Benchmark39.7%61.3%
ARC-AGI-151.5%5.5%
LMArena Hard Prompts13811434
Epoch Capabilities Index145.81138.33
SimpleBench—36.1%
CritPt9.3%—
Chess Puzzles30%—
Mystery Game Puzzles9%—
DTBench80.3%—
LMCA36.9%—
ForecastBench57.3—

Math GPT-5.4 nano leads

GPT-5.4 nano: 40.9 (#88), Grok 3: 38.0 (#145)

BenchmarkGPT-5.4 nanoGrok 3
OTIS Mock AIME 2024-202587.8%55.6%
LMArena Math14061391
FrontierMath (Feb 2025 set)25.9%3.8%
FrontierMath Tier 4 (v1)6.3%0%
FrontierMath (Tiers 1-3)44.9%—
FrontierMath Tier 412.2%—
ProofBench5%—
Omni-MATH—46.4%
MATH Level 5—88.7%

Knowledge Grok 3 leads

GPT-5.4 nano: 41.9 (#103), Grok 3: 46.2 (#82)

BenchmarkGPT-5.4 nanoGrok 3
GPQA Diamond78.5%75.8%
Vectara Hallucination Rate3.1%5.8%
LMArena Expert13961421
SimpleQA Verified11.7%—
MMLU-Pro—78.8%
Confabulations—14.2%
GPQA (HELM)—65%

Multimodal Not comparable

GPT-5.4 nano: 36.7 (#78), Grok 3: —

BenchmarkGPT-5.4 nanoGrok 3
LMArena Vision1196—

Multilingual Grok 3 leads

GPT-5.4 nano: 48.6 (#140), Grok 3: 52.3 (#87)

BenchmarkGPT-5.4 nanoGrok 3
LMArena Non-English13591410
LMArena Chinese13921448
LMArena French13961460
LMArena German13671431
LMArena Japanese13431387
LMArena Korean13201373
LMArena Russian13631416
LMArena Spanish13711417

Instruction Following Grok 3 leads

GPT-5.4 nano: 71.9 (#144), Grok 3: 75.0 (#73)

BenchmarkGPT-5.4 nanoGrok 3
LMArena Instruction Following13621409
IFEval—88.4%

Long Context GPT-5.4 nano leads

GPT-5.4 nano: 41.6 (#137), Grok 3: 38.7 (#192)

BenchmarkGPT-5.4 nanoGrok 3
LMArena Longer Query13661439
Fiction.LiveBench—58.3%

Writing & Preference Too close to call

GPT-5.4 nano: 55.7 (#142), Grok 3: 55.8 (#141)

BenchmarkGPT-5.4 nanoGrok 3
LMArena Text13721426
LMArena Creative Writing13141414
LMArena Multi-Turn13821425
Short-Story Creative Writing—76.4%
EQ-Bench Creative Writing—1186
WildBench—84.9%

Frequently asked questions

Is GPT-5.4 nano better than Grok 3?

GPT-5.4 nano is the stronger model overall, scoring 41.9 to 39.9 on the Noometry Index.

Is GPT-5.4 nano or Grok 3 better for coding?

GPT-5.4 nano scores higher on coding benchmarks: 43.6 versus 41.9 in the Noometry coding category.

How many benchmarks do GPT-5.4 nano and Grok 3 share?

27 benchmarks have published results for both models. GPT-5.4 nano has 40 scored results on Noometry and Grok 3 has 40.

Related comparisons

Go deeper