Noometry

Model comparison

GPT-5.4 nano vs Grok-2 (Dec 2024)

GPT-5.4 nano is the stronger model overall, scoring 41.9 to 33.7 on the Noometry Index.

Last verified . 23 shared benchmarks.

GPT-5.4 nano OpenAI

41.9

Rank #125 Confirmed

Grok-2 (Dec 2024) xAI

33.7

Rank #239 Confirmed

Summary

  • They share 23 benchmarks with published results for both. GPT-5.4 nano scores higher in 8 categories and Grok-2 (Dec 2024) in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where GPT-5.4 nano leads 40.9 to 20.8.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 87.8% for GPT-5.4 nano and 11.5% for Grok-2 (Dec 2024).

Side by side

GPT-5.4 nanoGrok-2 (Dec 2024)
ProviderOpenAIxAI
Noometry Index41.933.7
Released2026-03-172024-08-13
WeightsProprietaryProprietary
Context window400K—
Max output128K—
Input $ / M tokens$0.20—
Output $ / M tokens$1.25—
Results tracked4034

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.4 nano leads

GPT-5.4 nano: 43.6 (#84), Grok-2 (Dec 2024): 33.3 (#258)

BenchmarkGPT-5.4 nanoGrok-2 (Dec 2024)
WeirdML49.2%22.2%
LMArena Coding14051287
SciCode46.9%—
LiveBench Coding—46.4%
ALE-Bench1,005—

Reasoning GPT-5.4 nano leads

GPT-5.4 nano: 23.7 (#173), Grok-2 (Dec 2024): 16.9 (#299)

BenchmarkGPT-5.4 nanoGrok-2 (Dec 2024)
LMArena Hard Prompts13811272
DTBench80.3%65.2%
Epoch Capabilities Index145.81130.48
ARC-AGI-25.7%—
SimpleBench—22.7%
Kagi LLM Benchmark39.7%—
ARC-AGI-151.5%—
CritPt9.3%—
Chess Puzzles30%—
LiveBench Reasoning—54.8%
Mystery Game Puzzles9%—
LiveBench Data Analysis—54.5%
LMCA36.9%—
ForecastBench57.3—
LiveBench—54.3%

Math GPT-5.4 nano leads

GPT-5.4 nano: 40.9 (#88), Grok-2 (Dec 2024): 20.8 (#284)

BenchmarkGPT-5.4 nanoGrok-2 (Dec 2024)
OTIS Mock AIME 2024-202587.8%11.5%
LMArena Math14061283
FrontierMath (Feb 2025 set)25.9%0.7%
FrontierMath (Tiers 1-3)44.9%—
FrontierMath Tier 412.2%—
ProofBench5%—
LiveBench Math—54.9%
MATH Level 5—63.5%
FrontierMath Tier 4 (v1)6.3%—

Knowledge GPT-5.4 nano leads

GPT-5.4 nano: 41.9 (#103), Grok-2 (Dec 2024): 29.8 (#233)

BenchmarkGPT-5.4 nanoGrok-2 (Dec 2024)
GPQA Diamond78.5%53.8%
LMArena Expert13961254
SimpleQA Verified11.7%—
Confabulations—20.1%
Vectara Hallucination Rate3.1%—

Multimodal Not comparable

GPT-5.4 nano: 36.7 (#78), Grok-2 (Dec 2024): —

BenchmarkGPT-5.4 nanoGrok-2 (Dec 2024)
LMArena Vision1196—

Multilingual GPT-5.4 nano leads

GPT-5.4 nano: 48.6 (#140), Grok-2 (Dec 2024): 43.1 (#188)

BenchmarkGPT-5.4 nanoGrok-2 (Dec 2024)
LMArena Non-English13591282
LMArena Chinese13921289
LMArena French13961318
LMArena German13671287
LMArena Japanese13431244
LMArena Korean13201237
LMArena Russian13631286
LMArena Spanish13711281

Instruction Following GPT-5.4 nano leads

GPT-5.4 nano: 71.9 (#144), Grok-2 (Dec 2024): 66.9 (#202)

BenchmarkGPT-5.4 nanoGrok-2 (Dec 2024)
LMArena Instruction Following13621270
LiveBench Instruction Following—69.6%

Long Context GPT-5.4 nano leads

GPT-5.4 nano: 41.6 (#137), Grok-2 (Dec 2024): 38.8 (#190)

BenchmarkGPT-5.4 nanoGrok-2 (Dec 2024)
LMArena Longer Query13661276

Writing & Preference GPT-5.4 nano leads

GPT-5.4 nano: 55.7 (#142), Grok-2 (Dec 2024): 48.6 (#198)

BenchmarkGPT-5.4 nanoGrok-2 (Dec 2024)
LMArena Text13721305
LMArena Creative Writing13141284
LMArena Multi-Turn13821290
Short-Story Creative Writing—63.6%
LiveBench Language—45.6%

Frequently asked questions

Is GPT-5.4 nano better than Grok-2 (Dec 2024)?

GPT-5.4 nano is the stronger model overall, scoring 41.9 to 33.7 on the Noometry Index.

Is GPT-5.4 nano or Grok-2 (Dec 2024) better for coding?

GPT-5.4 nano scores higher on coding benchmarks: 43.6 versus 33.3 in the Noometry coding category.

How many benchmarks do GPT-5.4 nano and Grok-2 (Dec 2024) share?

23 benchmarks have published results for both models. GPT-5.4 nano has 40 scored results on Noometry and Grok-2 (Dec 2024) has 34.

Related comparisons

Go deeper