Noometry

Model comparison

GPT-4.1 mini vs Grok 4 Fast

Grok 4 Fast is the stronger model overall, scoring 39.4 to 33.6 on the Noometry Index.

Last verified . 24 shared benchmarks.

GPT-4.1 mini OpenAI

33.6

Rank #240 Confirmed

Grok 4 Fast xAI

39.4

Rank #167 Confirmed

Summary

  • They share 24 benchmarks with published results for both. GPT-4.1 mini scores higher in 3 categories and Grok 4 Fast in 6 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in long context, where Grok 4 Fast leads 63.2 to 31.8.
  • The biggest single-benchmark swing is Fiction.LiveBench: 44.4% for GPT-4.1 mini and 94.4% for Grok 4 Fast.

Side by side

GPT-4.1 miniGrok 4 Fast
ProviderOpenAIxAI
Noometry Index33.639.4
Released2025-04-142025-09-19
WeightsProprietaryProprietary
Context window1.05M—
Max output33K—
Input $ / M tokens$0.40—
Output $ / M tokens$1.60—
Results tracked4730

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok 4 Fast leads

GPT-4.1 mini: 30.6 (#293), Grok 4 Fast: 32.5 (#271)

BenchmarkGPT-4.1 miniGrok 4 Fast
WeirdML37.6%42.9%
LMArena Coding13671429
SWE-bench Verified (bash only)23.9%—
Aider Polyglot32.4%—
LMArena WebDev—1159
SciCode40.4%—
BigCodeBench Instruct48.9%—
CadEval16%—

Agentic & Tool Use GPT-4.1 mini leads

GPT-4.1 mini: 33.3 (#55), Grok 4 Fast: 29.5 (#86)

BenchmarkGPT-4.1 miniGrok 4 Fast
Berkeley Function Calling Leaderboard50.5%—
τ²-bench Banking—15.7%
Cybench—30%
LMArena Search—1171

Reasoning Grok 4 Fast leads

GPT-4.1 mini: 10.8 (#340), Grok 4 Fast: 22.2 (#201)

BenchmarkGPT-4.1 miniGrok 4 Fast
ARC-AGI-20%5.3%
Kagi LLM Benchmark48.6%66.1%
ARC-AGI-13.5%48.5%
LMArena Hard Prompts13491412
DTBench68.8%82.7%
Epoch Capabilities Index135.01144.2
CritPt0%—
Chess Puzzles7%—
Mystery Game Puzzles7%—
LMCA21.1%—
ForecastBench—60.5

Math Grok 4 Fast leads

GPT-4.1 mini: 24.1 (#270), Grok 4 Fast: 38.9 (#123)

BenchmarkGPT-4.1 miniGrok 4 Fast
LMArena Math13431419
FrontierMath (Tiers 1-3)6.7%—
OTIS Mock AIME 2024-202544.7%—
Omni-MATH49.1%—
MATH Level 587.3%—
FrontierMath (Feb 2025 set)4.5%—

Knowledge GPT-4.1 mini leads

GPT-4.1 mini: 34.7 (#194), Grok 4 Fast: 32.0 (#214)

BenchmarkGPT-4.1 miniGrok 4 Fast
LMArena Expert13381411
GPQA Diamond65.8%—
SimpleQA Verified12.7%—
MMLU-Pro78.3%—
Vectara Hallucination Rate—19.7%
GPQA (HELM)61.4%—

Multimodal Not comparable

GPT-4.1 mini: 35.8 (#82), Grok 4 Fast: —

BenchmarkGPT-4.1 miniGrok 4 Fast
LMArena Vision1181—

Multilingual Grok 4 Fast leads

GPT-4.1 mini: 45.7 (#166), Grok 4 Fast: 51.3 (#111)

BenchmarkGPT-4.1 miniGrok 4 Fast
LMArena Non-English13181396
LMArena Chinese13291457
LMArena French13581431
LMArena German13511383
LMArena Japanese12901352
LMArena Korean12981357
LMArena Russian13241389
LMArena Spanish13191415

Instruction Following Too close to call

GPT-4.1 mini: 73.7 (#118), Grok 4 Fast: 73.2 (#121)

BenchmarkGPT-4.1 miniGrok 4 Fast
LMArena Instruction Following13331387
IFEval90.4%—

Long Context Grok 4 Fast leads

GPT-4.1 mini: 31.8 (#275), Grok 4 Fast: 63.2 (#3)

BenchmarkGPT-4.1 miniGrok 4 Fast
Fiction.LiveBench44.4%94.4%
LMArena Longer Query13441415

Writing & Preference Grok 4 Fast leads

GPT-4.1 mini: 48.6 (#199), Grok 4 Fast: 60.0 (#102)

BenchmarkGPT-4.1 miniGrok 4 Fast
LMArena Text13401407
LMArena Creative Writing13001387
LMArena Multi-Turn13541414
EQ-Bench Creative Writing1147—
WildBench83.8%—

Frequently asked questions

Is GPT-4.1 mini better than Grok 4 Fast?

Grok 4 Fast is the stronger model overall, scoring 39.4 to 33.6 on the Noometry Index.

Is GPT-4.1 mini or Grok 4 Fast better for coding?

Grok 4 Fast scores higher on coding benchmarks: 32.5 versus 30.6 in the Noometry coding category.

How many benchmarks do GPT-4.1 mini and Grok 4 Fast share?

24 benchmarks have published results for both models. GPT-4.1 mini has 47 scored results on Noometry and Grok 4 Fast has 30.

Related comparisons

Go deeper