Noometry

Model comparison

Claude 3.5 Haiku vs DeepSeek V4 Flash

DeepSeek V4 Flash is the stronger model overall, scoring 53.6 to 29.2 on the Noometry Index.

Last verified . 25 shared benchmarks.

Claude 3.5 Haiku Anthropic

29.2

Rank #315 Confirmed

DeepSeek V4 Flash DeepSeek

53.6

Rank #35 Confirmed

Summary

  • They share 25 benchmarks with published results for both. Claude 3.5 Haiku scores higher in 0 categories and DeepSeek V4 Flash in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where DeepSeek V4 Flash leads 60.3 to 14.7.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 4.3% for Claude 3.5 Haiku and 94.4% for DeepSeek V4 Flash.
  • DeepSeek V4 Flash has downloadable open weights; the other is API-only.

Side by side

Claude 3.5 HaikuDeepSeek V4 Flash
ProviderAnthropicDeepSeek
Noometry Index29.253.6
Released2024-10-222026-04-24
WeightsProprietaryOpen
Context window—1M
Max output—393K
Input $ / M tokens—$0.15
Output $ / M tokens—$0.60
Results tracked4941

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding DeepSeek V4 Flash leads

Claude 3.5 Haiku: 32.9 (#265), DeepSeek V4 Flash: 47.9 (#59)

BenchmarkClaude 3.5 HaikuDeepSeek V4 Flash
SciCode27.4%49.9%
WeirdML30.7%63%
LMArena Coding12861457
FrontierCode—18.8%
Aider Polyglot28%—
LMArena WebDev—1582
BigCodeBench Instruct46.1%—
LiveBench Coding51.4%—
BigCodeBench Complete59%—
CadEval32%—
ALE-Bench—1,306

Agentic & Tool Use Not comparable

Claude 3.5 Haiku: 28.0 (#95), DeepSeek V4 Flash: —

BenchmarkClaude 3.5 HaikuDeepSeek V4 Flash
BALROG19.3%—

Reasoning DeepSeek V4 Flash leads

Claude 3.5 Haiku: 17.7 (#290), DeepSeek V4 Flash: 53.7 (#30)

BenchmarkClaude 3.5 HaikuDeepSeek V4 Flash
CritPt0%16.6%
LMArena Hard Prompts12511444
DTBench56.7%90.9%
Epoch Capabilities Index127.15154.49
ARC-AGI-2—61.4%
SimpleBench—61.1%
Kagi LLM Benchmark—52.2%
NYT Connections (extended)—89.6%
ARC-AGI-1—89%
Chess Puzzles—33%
LiveBench Reasoning28.1%—
Mystery Game Puzzles—34%
LiveBench Data Analysis48.5%—
LMCA—41.7%
LiveBench43.5%—

Math DeepSeek V4 Flash leads

Claude 3.5 Haiku: 14.7 (#300), DeepSeek V4 Flash: 60.3 (#37)

BenchmarkClaude 3.5 HaikuDeepSeek V4 Flash
OTIS Mock AIME 2024-20254.3%94.4%
LMArena Math12441427
FrontierMath (Tiers 1-3)—57.5%
FrontierMath Tier 4—24.4%
MathArena Final-Answer Competitions—76.5%
ProofBench—56%
Omni-MATH22.4%—
LiveBench Math35.5%—
MATH Level 546.4%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge DeepSeek V4 Flash leads

Claude 3.5 Haiku: 18.7 (#281), DeepSeek V4 Flash: 55.4 (#48)

BenchmarkClaude 3.5 HaikuDeepSeek V4 Flash
GPQA Diamond38.1%91%
LMArena Expert12081441
SimpleQA Verified—33.6%
MMLU-Pro60.5%—
Confabulations36.7%—
GPQA (HELM)36.3%—
MMLU74.3%—

Multimodal Not comparable

Claude 3.5 Haiku: 26.8 (#117), DeepSeek V4 Flash: —

BenchmarkClaude 3.5 HaikuDeepSeek V4 Flash
LMArena Vision1092—
GeoBench34%—

Multilingual DeepSeek V4 Flash leads

Claude 3.5 Haiku: 40.0 (#218), DeepSeek V4 Flash: 53.0 (#72)

BenchmarkClaude 3.5 HaikuDeepSeek V4 Flash
LMArena Non-English12381420
LMArena Chinese12291468
LMArena French12641439
LMArena German12371418
LMArena Japanese11751406
LMArena Korean11731384
LMArena Russian12531428
LMArena Spanish12611436

Instruction Following DeepSeek V4 Flash leads

Claude 3.5 Haiku: 62.9 (#234), DeepSeek V4 Flash: 74.9 (#81)

BenchmarkClaude 3.5 HaikuDeepSeek V4 Flash
LMArena Instruction Following12411421
LiveBench Instruction Following61.9%—
IFEval79.2%—

Long Context DeepSeek V4 Flash leads

Claude 3.5 Haiku: 38.3 (#200), DeepSeek V4 Flash: 43.8 (#85)

BenchmarkClaude 3.5 HaikuDeepSeek V4 Flash
LMArena Longer Query12611434

Writing & Preference DeepSeek V4 Flash leads

Claude 3.5 Haiku: 42.7 (#234), DeepSeek V4 Flash: 63.8 (#61)

BenchmarkClaude 3.5 HaikuDeepSeek V4 Flash
LMArena Text12551432
LMArena Creative Writing12331403
EQ-Bench Creative Writing11461559
LMArena Multi-Turn12651449
Short-Story Creative Writing73.5%—
WildBench76%—
LiveBench Language35.4%—

Frequently asked questions

Is Claude 3.5 Haiku better than DeepSeek V4 Flash?

DeepSeek V4 Flash is the stronger model overall, scoring 53.6 to 29.2 on the Noometry Index.

Is Claude 3.5 Haiku or DeepSeek V4 Flash better for coding?

DeepSeek V4 Flash scores higher on coding benchmarks: 47.9 versus 32.9 in the Noometry coding category.

How many benchmarks do Claude 3.5 Haiku and DeepSeek V4 Flash share?

25 benchmarks have published results for both models. Claude 3.5 Haiku has 49 scored results on Noometry and DeepSeek V4 Flash has 41.

Related comparisons

Go deeper