Noometry

Model comparison

Claude Haiku 4.5 vs Gemini 3.1 Pro Preview

Gemini 3.1 Pro Preview is the stronger model overall, scoring 56.7 to 39.3 on the Noometry Index. Claude Haiku 4.5 costs 2.3× less per token, which makes it the better buy when Gemini 3.1 Pro Preview's lead doesn't matter for your workload.

Last verified . 44 shared benchmarks.

Claude Haiku 4.5 Anthropic

39.3

Rank #155 Confirmed

Gemini 3.1 Pro Preview Google

56.7

Rank #20 Confirmed

Summary

  • They share 44 benchmarks with published results for both. Claude Haiku 4.5 scores higher in 1 category and Gemini 3.1 Pro Preview in 9 categories; 10 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Gemini 3.1 Pro Preview leads 71.7 to 15.1.
  • The biggest single-benchmark swing is NYT Connections (extended): 14.3% for Claude Haiku 4.5 and 97.4% for Gemini 3.1 Pro Preview.
  • Claude Haiku 4.5 is cheaper at $1 / $5 per million input/output tokens, against $2 / $12 for Gemini 3.1 Pro Preview.
  • Gemini 3.1 Pro Preview accepts more context: 1.05M tokens versus 200K.

Side by side

Claude Haiku 4.5Gemini 3.1 Pro Preview
ProviderAnthropicGoogle
Noometry Index39.356.7
Released2025-10-152026-02-19
WeightsProprietaryProprietary
Context window200K1.05M
Max output64K66K
Input $ / M tokens$1$2
Output $ / M tokens$5$12
Results tracked5371

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude Haiku 4.5 leads

Claude Haiku 4.5: 43.9 (#71), Gemini 3.1 Pro Preview: 42.7 (#88)

BenchmarkClaude Haiku 4.5Gemini 3.1 Pro Preview
LMArena WebDev13301446
SciCode43.3%58.9%
WeirdML45.4%72.1%
LMArena Coding14541483
ALE-Bench653.481,161
SWE-bench Verified—75.6%
DeepSWE—11.7%
SWE-bench Verified (bash only)66.6%—
SWE-bench Multilingual64.7%—
GSO—22.6%
MirrorCode—8.9%
AlgoTune—2.02

Agentic & Tool Use Gemini 3.1 Pro Preview leads

Claude Haiku 4.5: 33.5 (#50), Gemini 3.1 Pro Preview: 37.7 (#33)

BenchmarkClaude Haiku 4.5Gemini 3.1 Pro Preview
Terminal-Bench35.5%80.2%
DeepResearch Bench45.5%47.8%
BALROG31.2%57%
ExploitBench13.7%26.1%
Vending-Bench 2458.893,774
APEX-Agents—35.3%
Berkeley Function Calling Leaderboard68.7%—
τ²-bench Banking—26%
PostTrainBench—22%
GBAEval—0.8%
GDP.pdf—17%
LMArena Search—1211
METR Time Horizons—77%

Reasoning Gemini 3.1 Pro Preview leads

Claude Haiku 4.5: 15.1 (#310), Gemini 3.1 Pro Preview: 71.7 (#9)

BenchmarkClaude Haiku 4.5Gemini 3.1 Pro Preview
ARC-AGI-24%77.1%
NYT Connections (extended)14.3%97.4%
ARC-AGI-147.7%98%
CritPt0%17.7%
Chess Puzzles8%55%
LMArena Hard Prompts14201484
DTBench73.6%97.1%
LMCA30.9%53.8%
Epoch Capabilities Index142.42154.92
ForecastBench61.459
SimpleBench—79.6%
EnigmaEval—36.8%
Thematic Generalization—79.4%
EBR-Bench—14.3%
Mystery Game Puzzles—34%

Math Gemini 3.1 Pro Preview leads

Claude Haiku 4.5: 43.0 (#69), Gemini 3.1 Pro Preview: 61.8 (#28)

BenchmarkClaude Haiku 4.5Gemini 3.1 Pro Preview
OTIS Mock AIME 2024-202566.7%95.6%
LMArena Math13951485
FrontierMath (Feb 2025 set)5.9%36.9%
FrontierMath Tier 4 (v1)2.1%16.7%
FrontierMath (Tiers 1-3)—59.6%
FrontierMath Tier 4—26.8%
MathArena Final-Answer Competitions—86.5%
ProofBench—26%
Omni-MATH56.1%—
MATH Level 596.4%—

Knowledge Gemini 3.1 Pro Preview leads

Claude Haiku 4.5: 37.6 (#146), Gemini 3.1 Pro Preview: 71.8 (#3)

BenchmarkClaude Haiku 4.5Gemini 3.1 Pro Preview
GPQA Diamond71.2%94.4%
SimpleQA Verified13.2%73.5%
Vectara Hallucination Rate9.8%10.4%
LMArena Expert14431484
Humanity's Last Exam—46.4%
MMLU-Pro77.7%—
GPQA (HELM)60.5%—

Multimodal Gemini 3.1 Pro Preview leads

Claude Haiku 4.5: 27.0 (#111), Gemini 3.1 Pro Preview: 38.1 (#60)

BenchmarkClaude Haiku 4.5Gemini 3.1 Pro Preview
Blueprint-Bench 20%26.5%
LMArena Document14201444
LMArena Vision—1296
Furniture Assembly—26.7%

Multilingual Gemini 3.1 Pro Preview leads

Claude Haiku 4.5: 50.1 (#124), Gemini 3.1 Pro Preview: 57.1 (#11)

BenchmarkClaude Haiku 4.5Gemini 3.1 Pro Preview
LMArena Non-English13761475
LMArena Chinese14151528
LMArena French14081488
LMArena German13771491
LMArena Japanese13391494
LMArena Korean13461458
LMArena Russian13801496
LMArena Spanish14201480

Instruction Following Gemini 3.1 Pro Preview leads

Claude Haiku 4.5: 71.6 (#144), Gemini 3.1 Pro Preview: 77.1 (#31)

BenchmarkClaude Haiku 4.5Gemini 3.1 Pro Preview
LMArena Instruction Following14141465
IFEval80.1%—

Long Context Gemini 3.1 Pro Preview leads

Claude Haiku 4.5: 43.6 (#86), Gemini 3.1 Pro Preview: 47.3 (#17)

BenchmarkClaude Haiku 4.5Gemini 3.1 Pro Preview
LMArena Longer Query14271482
CL-bench—20.8%
CL-bench Life—16.9%

Writing & Preference Gemini 3.1 Pro Preview leads

Claude Haiku 4.5: 58.1 (#119), Gemini 3.1 Pro Preview: 66.3 (#36)

BenchmarkClaude Haiku 4.5Gemini 3.1 Pro Preview
LMArena Text13961480
LMArena Creative Writing13711482
EQ-Bench 410641142
LMArena Multi-Turn14091486
EQ-Bench Creative Writing—1491
WildBench83.9%—

Frequently asked questions

Is Claude Haiku 4.5 better than Gemini 3.1 Pro Preview?

Gemini 3.1 Pro Preview is the stronger model overall, scoring 56.7 to 39.3 on the Noometry Index. Claude Haiku 4.5 costs 2.3× less per token, which makes it the better buy when Gemini 3.1 Pro Preview's lead doesn't matter for your workload.

Which is cheaper, Claude Haiku 4.5 or Gemini 3.1 Pro Preview?

Claude Haiku 4.5 is cheaper. It lists at $1 per million input tokens and $5 per million output tokens; Gemini 3.1 Pro Preview lists at $2 and $12.

Is Claude Haiku 4.5 or Gemini 3.1 Pro Preview better for coding?

Claude Haiku 4.5 scores higher on coding benchmarks: 43.9 versus 42.7 in the Noometry coding category.

Which has the bigger context window?

Gemini 3.1 Pro Preview does, with 1.05M tokens against 200K.

How many benchmarks do Claude Haiku 4.5 and Gemini 3.1 Pro Preview share?

44 benchmarks have published results for both models. Claude Haiku 4.5 has 53 scored results on Noometry and Gemini 3.1 Pro Preview has 71.

Related comparisons

Go deeper