Noometry

Model comparison

GPT-4o vs Grok 4.20 (Non-Reasoning)

Grok 4.20 (Non-Reasoning) is the stronger model overall, scoring 48.6 to 28.6 on the Noometry Index.

Last verified . 31 shared benchmarks.

GPT-4o OpenAI

28.6

Rank #324 Confirmed

Grok 4.20 (Non-Reasoning) xAI

48.6

Rank #54 Confirmed

Summary

  • They share 31 benchmarks with published results for both. GPT-4o scores higher in 1 category and Grok 4.20 (Non-Reasoning) in 9 categories; 10 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Grok 4.20 (Non-Reasoning) leads 52.3 to 9.4.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 6.4% for GPT-4o and 92.2% for Grok 4.20 (Non-Reasoning).
  • Grok 4.20 (Non-Reasoning) is cheaper at $1.25 / $2.50 per million input/output tokens, against $2.50 / $10 for GPT-4o.
  • Grok 4.20 (Non-Reasoning) accepts more context: 1M tokens versus 128K.

Side by side

GPT-4oGrok 4.20 (Non-Reasoning)
ProviderOpenAIxAI
Noometry Index28.648.6
Released2024-05-132026-02-17
WeightsProprietaryProprietary
Context window128K1M
Max output16K30K
Input $ / M tokens$2.50$1.25
Output $ / M tokens$10$2.50
Results tracked7246

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok 4.20 (Non-Reasoning) leads

GPT-4o: 24.8 (#328), Grok 4.20 (Non-Reasoning): 42.1 (#112)

BenchmarkGPT-4oGrok 4.20 (Non-Reasoning)
WeirdML25.1%52.3%
LMArena Coding12971459
SWE-bench Verified31%—
SWE-bench Verified (bash only)21.6%—
Aider Polyglot45.3%—
LMArena WebDev—1375
GSO0%—
BigCodeBench Instruct51.1%—
LiveBench Coding51.4%—
BigCodeBench Complete61.1%—
CadEval26%—
ALE-Bench—1,150
HumanEval+87.2%—
MBPP+72.2%—

Agentic & Tool Use Grok 4.20 (Non-Reasoning) leads

GPT-4o: 21.0 (#141), Grok 4.20 (Non-Reasoning): 34.4 (#46)

BenchmarkGPT-4oGrok 4.20 (Non-Reasoning)
LMArena Search10061189
Terminal-Bench—57.3%
GDPval9.9%—
TheAgentCompany8.6%—
τ²-bench Banking—18%
Cybench12.5%—
BALROG32.3%—
METR Time Horizons40.8%—
Vending-Bench 2—4,663

Reasoning Grok 4.20 (Non-Reasoning) leads

GPT-4o: 9.4 (#343), Grok 4.20 (Non-Reasoning): 52.3 (#32)

BenchmarkGPT-4oGrok 4.20 (Non-Reasoning)
ARC-AGI-20%65.1%
ARC-AGI-14.5%89.5%
Chess Puzzles13%24%
LMArena Hard Prompts12811451
DTBench64.5%90.1%
LMCA16.6%38.7%
Epoch Capabilities Index128.97151.98
ForecastBench57.761.4
SimpleBench17.8%—
Kagi LLM Benchmark—75%
NYT Connections (extended)—85.4%
CritPt0%—
EnigmaEval0.8%—
Thematic Generalization—63.8%
LiveBench Reasoning55.8%—
LiveBench Data Analysis60.9%—
LiveBench55.3%—

Math Grok 4.20 (Non-Reasoning) leads

GPT-4o: 10.6 (#312), Grok 4.20 (Non-Reasoning): 48.2 (#65)

BenchmarkGPT-4oGrok 4.20 (Non-Reasoning)
FrontierMath (Tiers 1-3)0.4%44.9%
OTIS Mock AIME 2024-20256.4%92.2%
LMArena Math12851455
FrontierMath Tier 4—17.1%
ProofBench—14%
Omni-MATH29.3%—
LiveBench Math49.5%—
MATH Level 553.3%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge Grok 4.20 (Non-Reasoning) leads

GPT-4o: 28.8 (#242), Grok 4.20 (Non-Reasoning): 52.8 (#60)

BenchmarkGPT-4oGrok 4.20 (Non-Reasoning)
GPQA Diamond49.2%89.3%
SimpleQA Verified26%30.2%
LMArena Expert12501439
Humanity's Last Exam2.7%—
MMLU-Pro71.3%—
Confabulations15.3%—
Vectara Hallucination Rate9.6%—
GPQA (HELM)52%—
MMLU88.1%—

Multimodal GPT-4o leads

GPT-4o: 34.5 (#91), Grok 4.20 (Non-Reasoning): 33.3 (#98)

BenchmarkGPT-4oGrok 4.20 (Non-Reasoning)
LMArena Vision11371263
Video-MME71.9%—
GeoBench71%—
VPCT40%—
Blueprint-Bench 2—0%
LMArena Document—1416
ScienceQA88.5%—

Multilingual Grok 4.20 (Non-Reasoning) leads

GPT-4o: 43.2 (#186), Grok 4.20 (Non-Reasoning): 54.5 (#40)

BenchmarkGPT-4oGrok 4.20 (Non-Reasoning)
LMArena Non-English12831441
LMArena Chinese12771481
LMArena French13041476
LMArena German12821465
LMArena Japanese12571449
LMArena Korean12341417
LMArena Russian12861458
LMArena Spanish12921443

Instruction Following Grok 4.20 (Non-Reasoning) leads

GPT-4o: 66.6 (#207), Grok 4.20 (Non-Reasoning): 74.8 (#83)

BenchmarkGPT-4oGrok 4.20 (Non-Reasoning)
LMArena Instruction Following12781420
LiveBench Instruction Following68.6%—
IFEval81.7%—

Long Context Grok 4.20 (Non-Reasoning) leads

GPT-4o: 39.4 (#179), Grok 4.20 (Non-Reasoning): 45.5 (#34)

BenchmarkGPT-4oGrok 4.20 (Non-Reasoning)
LMArena Longer Query12891437
Fiction.LiveBench66.7%—
CL-bench—22.2%
CL-bench Life—11.9%

Writing & Preference Grok 4.20 (Non-Reasoning) leads

GPT-4o: 52.6 (#166), Grok 4.20 (Non-Reasoning): 65.7 (#44)

BenchmarkGPT-4oGrok 4.20 (Non-Reasoning)
LMArena Text13001451
LMArena Creative Writing12921438
LMArena Multi-Turn13021456
Short-Story Creative Writing81.8%—
EQ-Bench Creative Writing—1574
WildBench82.8%—
LiveBench Language47.6%—

Frequently asked questions

Is GPT-4o better than Grok 4.20 (Non-Reasoning)?

Grok 4.20 (Non-Reasoning) is the stronger model overall, scoring 48.6 to 28.6 on the Noometry Index.

Which is cheaper, GPT-4o or Grok 4.20 (Non-Reasoning)?

Grok 4.20 (Non-Reasoning) is cheaper. It lists at $1.25 per million input tokens and $2.50 per million output tokens; GPT-4o lists at $2.50 and $10.

Is GPT-4o or Grok 4.20 (Non-Reasoning) better for coding?

Grok 4.20 (Non-Reasoning) scores higher on coding benchmarks: 42.1 versus 24.8 in the Noometry coding category.

Which has the bigger context window?

Grok 4.20 (Non-Reasoning) does, with 1M tokens against 128K.

How many benchmarks do GPT-4o and Grok 4.20 (Non-Reasoning) share?

31 benchmarks have published results for both models. GPT-4o has 72 scored results on Noometry and Grok 4.20 (Non-Reasoning) has 46.

Related comparisons

Go deeper