Noometry

Model comparison

Claude Haiku 4.5 vs Grok 4.20 (Non-Reasoning)

Grok 4.20 (Non-Reasoning) is the stronger model overall, scoring 48.5 to 39.3 on the Noometry Index.

Last verified . 35 shared benchmarks.

Claude Haiku 4.5 Anthropic

39.3

Rank #155 Confirmed

Grok 4.20 (Non-Reasoning) xAI

48.5

Rank #48 Confirmed

Summary

  • They share 35 benchmarks with published results for both. Claude Haiku 4.5 scores higher in 1 category and Grok 4.20 (Non-Reasoning) in 9 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Grok 4.20 (Non-Reasoning) leads 52.2 to 15.1.
  • The biggest single-benchmark swing is NYT Connections (extended): 14.3% for Claude Haiku 4.5 and 85.4% for Grok 4.20 (Non-Reasoning).
  • Grok 4.20 (Non-Reasoning) is cheaper at $1.25 / $2.50 per million input/output tokens, against $1 / $5 for Claude Haiku 4.5.
  • Grok 4.20 (Non-Reasoning) accepts more context: 1M tokens versus 200K.

Side by side

Claude Haiku 4.5Grok 4.20 (Non-Reasoning)
ProviderAnthropicxAI
Noometry Index39.348.5
Released2025-10-152026-02-17
WeightsProprietaryProprietary
Context window200K1M
Max output64K30K
Input $ / M tokens$1$1.25
Output $ / M tokens$5$2.50
Results tracked5346

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude Haiku 4.5 leads

Claude Haiku 4.5: 43.9 (#71), Grok 4.20 (Non-Reasoning): 41.8 (#104)

BenchmarkClaude Haiku 4.5Grok 4.20 (Non-Reasoning)
LMArena WebDev13301374
WeirdML45.4%52.3%
LMArena Coding14541458
ALE-Bench653.481,150
SWE-bench Verified (bash only)66.6%—
SWE-bench Multilingual64.7%—
SciCode43.3%—

Agentic & Tool Use Too close to call

Claude Haiku 4.5: 33.5 (#50), Grok 4.20 (Non-Reasoning): 34.3 (#43)

BenchmarkClaude Haiku 4.5Grok 4.20 (Non-Reasoning)
Terminal-Bench35.5%57.3%
Vending-Bench 2458.894,663
Berkeley Function Calling Leaderboard68.7%—
τ²-bench Banking—18%
DeepResearch Bench45.5%—
BALROG31.2%—
ExploitBench13.7%—
LMArena Search—1189

Reasoning Grok 4.20 (Non-Reasoning) leads

Claude Haiku 4.5: 15.1 (#310), Grok 4.20 (Non-Reasoning): 52.2 (#30)

BenchmarkClaude Haiku 4.5Grok 4.20 (Non-Reasoning)
ARC-AGI-24%65.1%
NYT Connections (extended)14.3%85.4%
ARC-AGI-147.7%89.5%
Chess Puzzles8%24%
LMArena Hard Prompts14201451
DTBench73.6%90.1%
LMCA30.9%38.7%
Epoch Capabilities Index142.42152.01
ForecastBench61.461.4
Kagi LLM Benchmark—75%
CritPt0%—
Thematic Generalization—63.8%

Math Grok 4.20 (Non-Reasoning) leads

Claude Haiku 4.5: 43.0 (#69), Grok 4.20 (Non-Reasoning): 48.0 (#54)

BenchmarkClaude Haiku 4.5Grok 4.20 (Non-Reasoning)
OTIS Mock AIME 2024-202566.7%92.2%
LMArena Math13951455
FrontierMath (Tiers 1-3)—44.9%
FrontierMath Tier 4—17.1%
ProofBench—14%
Omni-MATH56.1%—
MATH Level 596.4%—
FrontierMath (Feb 2025 set)5.9%—
FrontierMath Tier 4 (v1)2.1%—

Knowledge Grok 4.20 (Non-Reasoning) leads

Claude Haiku 4.5: 37.6 (#146), Grok 4.20 (Non-Reasoning): 52.8 (#54)

BenchmarkClaude Haiku 4.5Grok 4.20 (Non-Reasoning)
GPQA Diamond71.2%89.3%
SimpleQA Verified13.2%30.2%
LMArena Expert14431440
MMLU-Pro77.7%—
Vectara Hallucination Rate9.8%—
GPQA (HELM)60.5%—

Multimodal Grok 4.20 (Non-Reasoning) leads

Claude Haiku 4.5: 27.0 (#111), Grok 4.20 (Non-Reasoning): 33.2 (#88)

BenchmarkClaude Haiku 4.5Grok 4.20 (Non-Reasoning)
Blueprint-Bench 20%0%
LMArena Document14201416
LMArena Vision—1263

Multilingual Grok 4.20 (Non-Reasoning) leads

Claude Haiku 4.5: 50.1 (#124), Grok 4.20 (Non-Reasoning): 54.7 (#38)

BenchmarkClaude Haiku 4.5Grok 4.20 (Non-Reasoning)
LMArena Non-English13761440
LMArena Chinese14151481
LMArena French14081477
LMArena German13771463
LMArena Japanese13391450
LMArena Korean13461417
LMArena Russian13801457
LMArena Spanish14201442

Instruction Following Grok 4.20 (Non-Reasoning) leads

Claude Haiku 4.5: 71.6 (#144), Grok 4.20 (Non-Reasoning): 75.0 (#79)

BenchmarkClaude Haiku 4.5Grok 4.20 (Non-Reasoning)
LMArena Instruction Following14141420
IFEval80.1%—

Long Context Grok 4.20 (Non-Reasoning) leads

Claude Haiku 4.5: 43.6 (#86), Grok 4.20 (Non-Reasoning): 45.4 (#33)

BenchmarkClaude Haiku 4.5Grok 4.20 (Non-Reasoning)
LMArena Longer Query14271436
CL-bench—22.2%
CL-bench Life—11.9%

Writing & Preference Grok 4.20 (Non-Reasoning) leads

Claude Haiku 4.5: 58.1 (#119), Grok 4.20 (Non-Reasoning): 65.9 (#42)

BenchmarkClaude Haiku 4.5Grok 4.20 (Non-Reasoning)
LMArena Text13961451
LMArena Creative Writing13711439
LMArena Multi-Turn14091456
EQ-Bench Creative Writing—1574
WildBench83.9%—
EQ-Bench 41064—

Frequently asked questions

Is Claude Haiku 4.5 better than Grok 4.20 (Non-Reasoning)?

Grok 4.20 (Non-Reasoning) is the stronger model overall, scoring 48.5 to 39.3 on the Noometry Index.

Which is cheaper, Claude Haiku 4.5 or Grok 4.20 (Non-Reasoning)?

Grok 4.20 (Non-Reasoning) is cheaper. It lists at $1.25 per million input tokens and $2.50 per million output tokens; Claude Haiku 4.5 lists at $1 and $5.

Is Claude Haiku 4.5 or Grok 4.20 (Non-Reasoning) better for coding?

Claude Haiku 4.5 scores higher on coding benchmarks: 43.9 versus 41.8 in the Noometry coding category.

Which has the bigger context window?

Grok 4.20 (Non-Reasoning) does, with 1M tokens against 200K.

How many benchmarks do Claude Haiku 4.5 and Grok 4.20 (Non-Reasoning) share?

35 benchmarks have published results for both models. Claude Haiku 4.5 has 53 scored results on Noometry and Grok 4.20 (Non-Reasoning) has 46.

Related comparisons

Go deeper