Noometry

Model comparison

Claude 3 Haiku vs Grok 4.1 Fast

Grok 4.1 Fast is the stronger model overall, scoring 40.9 to 25.7 on the Noometry Index.

Last verified . 21 shared benchmarks.

Claude 3 Haiku Anthropic

25.7

Rank #333 Confirmed

Grok 4.1 Fast xAI

40.9

Rank #131 Confirmed

Summary

  • They share 21 benchmarks with published results for both. Claude 3 Haiku scores higher in 0 categories and Grok 4.1 Fast in 9 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Grok 4.1 Fast leads 57.4 to 29.8.
  • The biggest single-benchmark swing is DTBench: 50.1% for Claude 3 Haiku and 87.7% for Grok 4.1 Fast.

Side by side

Claude 3 HaikuGrok 4.1 Fast
ProviderAnthropicxAI
Noometry Index25.740.9
Released2024-03-072025-06-27
WeightsProprietaryProprietary
Context window—128K
Max output—30K
Input $ / M tokens—$0.20
Output $ / M tokens—$0.50
Results tracked3732

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok 4.1 Fast leads

Claude 3 Haiku: 26.1 (#320), Grok 4.1 Fast: 33.8 (#236)

BenchmarkClaude 3 HaikuGrok 4.1 Fast
LMArena Coding11991411
LMArena WebDev—1241
WeirdML9.8%—
BigCodeBench Instruct39.4%—
BigCodeBench Complete50.1%—
CadEval12%—
ALE-Bench—394.93
HumanEval+68.9%—
MBPP+68.8%—

Agentic & Tool Use Not comparable

Claude 3 Haiku: —, Grok 4.1 Fast: 36.1 (#35)

BenchmarkClaude 3 HaikuGrok 4.1 Fast
Berkeley Function Calling Leaderboard—69.6%
τ²-bench Banking—13.1%
LMArena Search—1171
Vending-Bench 2—1,107

Reasoning Grok 4.1 Fast leads

Claude 3 Haiku: 15.7 (#303), Grok 4.1 Fast: 42.3 (#45)

BenchmarkClaude 3 HaikuGrok 4.1 Fast
LMArena Hard Prompts11741407
DTBench50.1%87.7%
ForecastBench53.261
SimpleBench—56%
Kagi LLM Benchmark34.2%—
NYT Connections (extended)—87.4%
LMCA8.8%—
Epoch Capabilities Index118.3—
WinoGrande74.2%—

Math Grok 4.1 Fast leads

Claude 3 Haiku: 9.2 (#313), Grok 4.1 Fast: 30.9 (#201)

BenchmarkClaude 3 HaikuGrok 4.1 Fast
LMArena Math11881408
MathArena Final-Answer Competitions—60.9%
OTIS Mock AIME 2024-20251.8%—
ProofBench—4%
MATH Level 514.9%—

Knowledge Grok 4.1 Fast leads

Claude 3 Haiku: 17.3 (#280), Grok 4.1 Fast: 32.6 (#203)

BenchmarkClaude 3 HaikuGrok 4.1 Fast
LMArena Expert11481398
GPQA Diamond36.3%—
Confabulations34.2%—
Vectara Hallucination Rate—17.8%
MMLU73.8%—

Multimodal Grok 4.1 Fast leads

Claude 3 Haiku: 23.1 (#123), Grok 4.1 Fast: 36.3 (#71)

BenchmarkClaude 3 HaikuGrok 4.1 Fast
LMArena Vision9501201
ScienceQA72%—

Multilingual Grok 4.1 Fast leads

Claude 3 Haiku: 36.1 (#238), Grok 4.1 Fast: 51.1 (#110)

BenchmarkClaude 3 HaikuGrok 4.1 Fast
LMArena Non-English11781391
LMArena Chinese11551441
LMArena French11941416
LMArena German11741404
LMArena Japanese11021347
LMArena Korean11081361
LMArena Russian12041387
LMArena Spanish11651411

Instruction Following Grok 4.1 Fast leads

Claude 3 Haiku: 61.5 (#242), Grok 4.1 Fast: 72.8 (#127)

BenchmarkClaude 3 HaikuGrok 4.1 Fast
LMArena Instruction Following11741376

Long Context Grok 4.1 Fast leads

Claude 3 Haiku: 36.1 (#232), Grok 4.1 Fast: 42.4 (#121)

BenchmarkClaude 3 HaikuGrok 4.1 Fast
LMArena Longer Query11911390

Writing & Preference Grok 4.1 Fast leads

Claude 3 Haiku: 29.8 (#286), Grok 4.1 Fast: 57.4 (#126)

BenchmarkClaude 3 HaikuGrok 4.1 Fast
LMArena Text11951408
LMArena Creative Writing11571394
EQ-Bench Creative Writing7171327
LMArena Multi-Turn11901389

Frequently asked questions

Is Claude 3 Haiku better than Grok 4.1 Fast?

Grok 4.1 Fast is the stronger model overall, scoring 40.9 to 25.7 on the Noometry Index.

Is Claude 3 Haiku or Grok 4.1 Fast better for coding?

Grok 4.1 Fast scores higher on coding benchmarks: 33.8 versus 26.1 in the Noometry coding category.

How many benchmarks do Claude 3 Haiku and Grok 4.1 Fast share?

21 benchmarks have published results for both models. Claude 3 Haiku has 37 scored results on Noometry and Grok 4.1 Fast has 32.

Related comparisons

Go deeper