Noometry

Model comparison

Claude 3 Haiku vs Nemotron 3 Super

Nemotron 3 Super is the stronger model overall, scoring 39.6 to 25.7 on the Noometry Index.

Last verified . 16 shared benchmarks.

Claude 3 Haiku Anthropic

25.7

Rank #333 Confirmed

Nemotron 3 Super NVIDIA

39.6

Rank #147 Confirmed

Summary

  • They share 16 benchmarks with published results for both. Claude 3 Haiku scores higher in 0 categories and Nemotron 3 Super in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Nemotron 3 Super leads 37.6 to 9.2.
  • The biggest single-benchmark swing is WeirdML: 9.8% for Claude 3 Haiku and 38% for Nemotron 3 Super.
  • Nemotron 3 Super has downloadable open weights; the other is API-only.

Side by side

Claude 3 HaikuNemotron 3 Super
ProviderAnthropicNVIDIA
Noometry Index25.739.6
Released2024-03-072026-03-11
WeightsProprietaryOpen
Context window—262K
Max output—131K
Input $ / M tokens—$0.08
Output $ / M tokens—$0.45
Results tracked3721

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Nemotron 3 Super leads

Claude 3 Haiku: 26.1 (#320), Nemotron 3 Super: 38.9 (#153)

BenchmarkClaude 3 HaikuNemotron 3 Super
WeirdML9.8%38%
LMArena Coding11991403
SciCode—36%
BigCodeBench Instruct39.4%—
BigCodeBench Complete50.1%—
CadEval12%—
ALE-Bench—213.9
HumanEval+68.9%—
MBPP+68.8%—

Reasoning Nemotron 3 Super leads

Claude 3 Haiku: 15.7 (#303), Nemotron 3 Super: 17.9 (#274)

BenchmarkClaude 3 HaikuNemotron 3 Super
LMArena Hard Prompts11741386
Kagi LLM Benchmark34.2%—
NYT Connections (extended)—15.4%
CritPt—3.1%
DTBench50.1%—
LMCA8.8%—
Epoch Capabilities Index118.3—
ForecastBench53.2—
WinoGrande74.2%—

Math Nemotron 3 Super leads

Claude 3 Haiku: 9.2 (#313), Nemotron 3 Super: 37.6 (#95)

BenchmarkClaude 3 HaikuNemotron 3 Super
LMArena Math11881381
MathArena Final-Answer Competitions—60.4%
OTIS Mock AIME 2024-20251.8%—
MATH Level 514.9%—

Knowledge Nemotron 3 Super leads

Claude 3 Haiku: 17.3 (#280), Nemotron 3 Super: 38.2 (#136)

BenchmarkClaude 3 HaikuNemotron 3 Super
LMArena Expert11481397
GPQA Diamond36.3%—
Confabulations34.2%—
MMLU73.8%—

Multimodal Not comparable

Claude 3 Haiku: 23.1 (#123), Nemotron 3 Super: —

BenchmarkClaude 3 HaikuNemotron 3 Super
LMArena Vision950—
ScienceQA72%—

Multilingual Nemotron 3 Super leads

Claude 3 Haiku: 36.1 (#238), Nemotron 3 Super: 48.6 (#139)

BenchmarkClaude 3 HaikuNemotron 3 Super
LMArena Non-English11781356
LMArena Chinese11551430
LMArena French11941404
LMArena German11741341
LMArena Russian12041336
LMArena Spanish11651417
LMArena Japanese1102—
LMArena Korean1108—

Instruction Following Nemotron 3 Super leads

Claude 3 Haiku: 61.5 (#242), Nemotron 3 Super: 71.4 (#149)

BenchmarkClaude 3 HaikuNemotron 3 Super
LMArena Instruction Following11741347

Long Context Nemotron 3 Super leads

Claude 3 Haiku: 36.1 (#232), Nemotron 3 Super: 41.5 (#134)

BenchmarkClaude 3 HaikuNemotron 3 Super
LMArena Longer Query11911362

Writing & Preference Nemotron 3 Super leads

Claude 3 Haiku: 29.8 (#286), Nemotron 3 Super: 56.1 (#135)

BenchmarkClaude 3 HaikuNemotron 3 Super
LMArena Text11951378
LMArena Creative Writing11571317
LMArena Multi-Turn11901369
EQ-Bench Creative Writing717—

Frequently asked questions

Is Claude 3 Haiku better than Nemotron 3 Super?

Nemotron 3 Super is the stronger model overall, scoring 39.6 to 25.7 on the Noometry Index.

Is Claude 3 Haiku or Nemotron 3 Super better for coding?

Nemotron 3 Super scores higher on coding benchmarks: 38.9 versus 26.1 in the Noometry coding category.

How many benchmarks do Claude 3 Haiku and Nemotron 3 Super share?

16 benchmarks have published results for both models. Claude 3 Haiku has 37 scored results on Noometry and Nemotron 3 Super has 21.

Related comparisons

Go deeper