Noometry

Model comparison

Claude Haiku 4.5 vs Gemini 2.5 Pro

Gemini 2.5 Pro is the stronger model overall, scoring 45.0 to 39.5 on the Noometry Index. Claude Haiku 4.5 costs 1.7× less per token, which makes it the better buy when Gemini 2.5 Pro's lead doesn't matter for your workload.

Last verified . 46 shared benchmarks.

Claude Haiku 4.5 Anthropic

39.5

Rank #165 Confirmed

Gemini 2.5 Pro Google

45.0

Rank #75 Confirmed

Summary

  • They share 46 benchmarks with published results for both. Claude Haiku 4.5 scores higher in 3 categories and Gemini 2.5 Pro in 7 categories; 10 gaps are clear of the uncertainty.
  • The widest gap is in multimodal, where Gemini 2.5 Pro leads 45.2 to 26.8.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 66.7% for Claude Haiku 4.5 and 84.7% for Gemini 2.5 Pro.
  • Claude Haiku 4.5 is cheaper at $1 / $5 per million input/output tokens, against $1.25 / $10 for Gemini 2.5 Pro.
  • Gemini 2.5 Pro accepts more context: 1.05M tokens versus 200K.

Side by side

Claude Haiku 4.5Gemini 2.5 Pro
ProviderAnthropicGoogle
Noometry Index39.545.0
Released2025-10-152025-03-25
WeightsProprietaryProprietary
Context window200K1.05M
Max output64K66K
Input $ / M tokens$1$1.25
Output $ / M tokens$5$10
Results tracked5378

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude Haiku 4.5 leads

Claude Haiku 4.5: 44.0 (#78), Gemini 2.5 Pro: 42.4 (#101)

BenchmarkClaude Haiku 4.5Gemini 2.5 Pro
SWE-bench Verified (bash only)66.6%53.6%
LMArena WebDev13301227
SciCode43.3%42.8%
WeirdML45.4%54%
LMArena Coding14531452
ALE-Bench653.48785.52
SWE-bench Verified—57.6%
Aider Polyglot—83.1%
SWE-bench Multilingual64.7%—
GSO—3.9%
LiveBench Coding—85.9%
CadEval—64%
AlgoTune—1.51

Agentic & Tool Use Claude Haiku 4.5 leads

Claude Haiku 4.5: 33.6 (#52), Gemini 2.5 Pro: 29.2 (#88)

BenchmarkClaude Haiku 4.5Gemini 2.5 Pro
Terminal-Bench35.5%32.6%
DeepResearch Bench45.5%42.8%
BALROG31.2%43.3%
Vending-Bench 2458.89573.64
Berkeley Function Calling Leaderboard68.7%—
GDPval—23.3%
Remote Labor Index—0.8%
TheAgentCompany—30.3%
τ²-bench Banking—13.7%
ExploitBench13.7%—
LMArena Search—1142
METR Time Horizons—55.4%

Reasoning Gemini 2.5 Pro leads

Claude Haiku 4.5: 15.1 (#320), Gemini 2.5 Pro: 28.8 (#99)

BenchmarkClaude Haiku 4.5Gemini 2.5 Pro
ARC-AGI-24%4.9%
ARC-AGI-147.7%41%
CritPt0%2%
Chess Puzzles8%20%
LMArena Hard Prompts14201455
DTBench73.6%82.4%
LMCA30.9%34.8%
Epoch Capabilities Index142.41145.32
ForecastBench61.461.3
SimpleBench—62.4%
Kagi LLM Benchmark—70.3%
NYT Connections (extended)14.3%—
EnigmaEval—5.6%
LiveBench Reasoning—89.8%
LiveBench Data Analysis—79.9%
LiveBench—82.3%

Math Claude Haiku 4.5 leads

Claude Haiku 4.5: 44.9 (#78), Gemini 2.5 Pro: 32.5 (#213)

BenchmarkClaude Haiku 4.5Gemini 2.5 Pro
OTIS Mock AIME 2024-202566.7%84.7%
Omni-MATH56.1%41.6%
LMArena Math13961450
MATH Level 596.4%95.9%
FrontierMath (Feb 2025 set)5.9%14.1%
FrontierMath Tier 4 (v1)2.1%4.2%
FrontierMath (Tiers 1-3)—24.6%
FrontierMath Tier 4—0%
LiveBench Math—90.2%

Knowledge Gemini 2.5 Pro leads

Claude Haiku 4.5: 37.7 (#153), Gemini 2.5 Pro: 56.0 (#46)

BenchmarkClaude Haiku 4.5Gemini 2.5 Pro
GPQA Diamond71.2%85.3%
MMLU-Pro77.7%86.3%
Vectara Hallucination Rate9.8%7%
GPQA (HELM)60.5%74.9%
LMArena Expert14421452
Humanity's Last Exam—21.6%
SimpleQA Verified13.2%—
Confabulations—10.6%

Multimodal Gemini 2.5 Pro leads

Claude Haiku 4.5: 26.8 (#118), Gemini 2.5 Pro: 45.2 (#18)

BenchmarkClaude Haiku 4.5Gemini 2.5 Pro
LMArena Document14201421
LMArena Vision—1263
GeoBench—86%
VPCT—48%
Blueprint-Bench 20%—
SpatialViz-Bench—44.7%

Multilingual Gemini 2.5 Pro leads

Claude Haiku 4.5: 49.9 (#129), Gemini 2.5 Pro: 55.3 (#31)

BenchmarkClaude Haiku 4.5Gemini 2.5 Pro
LMArena Non-English13771451
LMArena Chinese14171507
LMArena French14081472
LMArena German13751487
LMArena Japanese13391461
LMArena Korean13471434
LMArena Russian13811461
LMArena Spanish14201473

Instruction Following Gemini 2.5 Pro leads

Claude Haiku 4.5: 71.4 (#149), Gemini 2.5 Pro: 75.0 (#75)

BenchmarkClaude Haiku 4.5Gemini 2.5 Pro
IFEval80.1%84%
LMArena Instruction Following14141437
LiveBench Instruction Following—80.6%

Long Context Gemini 2.5 Pro leads

Claude Haiku 4.5: 43.6 (#92), Gemini 2.5 Pro: 59.8 (#5)

BenchmarkClaude Haiku 4.5Gemini 2.5 Pro
LMArena Longer Query14271449
Fiction.LiveBench—91.7%

Writing & Preference Gemini 2.5 Pro leads

Claude Haiku 4.5: 57.9 (#123), Gemini 2.5 Pro: 63.7 (#62)

BenchmarkClaude Haiku 4.5Gemini 2.5 Pro
LMArena Text13961458
LMArena Creative Writing13721454
WildBench83.9%85.7%
LMArena Multi-Turn14091453
Short-Story Creative Writing—83.8%
EQ-Bench Creative Writing—1421
EQ-Bench 41064—
LiveBench Language—67.8%

Frequently asked questions

Is Claude Haiku 4.5 better than Gemini 2.5 Pro?

Gemini 2.5 Pro is the stronger model overall, scoring 45.0 to 39.5 on the Noometry Index. Claude Haiku 4.5 costs 1.7× less per token, which makes it the better buy when Gemini 2.5 Pro's lead doesn't matter for your workload.

Which is cheaper, Claude Haiku 4.5 or Gemini 2.5 Pro?

Claude Haiku 4.5 is cheaper. It lists at $1 per million input tokens and $5 per million output tokens; Gemini 2.5 Pro lists at $1.25 and $10.

Is Claude Haiku 4.5 or Gemini 2.5 Pro better for coding?

Claude Haiku 4.5 scores higher on coding benchmarks: 44.0 versus 42.4 in the Noometry coding category.

Which has the bigger context window?

Gemini 2.5 Pro does, with 1.05M tokens against 200K.

How many benchmarks do Claude Haiku 4.5 and Gemini 2.5 Pro share?

46 benchmarks have published results for both models. Claude Haiku 4.5 has 53 scored results on Noometry and Gemini 2.5 Pro has 78.

Related comparisons

Go deeper