Noometry

Model comparison

Claude 3.5 Haiku vs Trinity Large Thinking

Trinity Large Thinking is the stronger model overall, scoring 38.6 to 29.2 on the Noometry Index.

Last verified . 19 shared benchmarks.

Claude 3.5 Haiku Anthropic

29.2

Rank #315 Confirmed

Trinity Large Thinking Arcee AI

38.6

Rank #185 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Claude 3.5 Haiku scores higher in 1 category and Trinity Large Thinking in 7 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Trinity Large Thinking leads 37.6 to 14.7.
  • The biggest single-benchmark swing is SciCode: 27.4% for Claude 3.5 Haiku and 36.1% for Trinity Large Thinking.
  • Trinity Large Thinking has downloadable open weights; the other is API-only.

Side by side

Claude 3.5 HaikuTrinity Large Thinking
ProviderAnthropicArcee AI
Noometry Index29.238.6
Released2024-10-222026-04-01
WeightsProprietaryOpen
Context window—262K
Max output—80K
Input $ / M tokens—$0.25
Output $ / M tokens—$0.80
Results tracked4924

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Trinity Large Thinking leads

Claude 3.5 Haiku: 32.9 (#265), Trinity Large Thinking: 34.1 (#244)

BenchmarkClaude 3.5 HaikuTrinity Large Thinking
SciCode27.4%36.1%
LMArena Coding12861381
Aider Polyglot28%—
LMArena WebDev—1238
WeirdML30.7%—
BigCodeBench Instruct46.1%—
LiveBench Coding51.4%—
BigCodeBench Complete59%—
CadEval32%—

Agentic & Tool Use Not comparable

Claude 3.5 Haiku: 28.0 (#95), Trinity Large Thinking: —

BenchmarkClaude 3.5 HaikuTrinity Large Thinking
BALROG19.3%—

Reasoning Too close to call

Claude 3.5 Haiku: 17.7 (#290), Trinity Large Thinking: 16.9 (#298)

BenchmarkClaude 3.5 HaikuTrinity Large Thinking
CritPt0%0.9%
LMArena Hard Prompts12511350
NYT Connections (extended)—16.5%
Thematic Generalization—41.6%
LiveBench Reasoning28.1%—
DTBench56.7%—
LiveBench Data Analysis48.5%—
Surface Evolver Bench—15.6%
Epoch Capabilities Index127.15—
LiveBench43.5%—

Math Trinity Large Thinking leads

Claude 3.5 Haiku: 14.7 (#300), Trinity Large Thinking: 37.6 (#149)

BenchmarkClaude 3.5 HaikuTrinity Large Thinking
LMArena Math12441366
OTIS Mock AIME 2024-20254.3%—
Omni-MATH22.4%—
LiveBench Math35.5%—
MATH Level 546.4%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge Trinity Large Thinking leads

Claude 3.5 Haiku: 18.7 (#281), Trinity Large Thinking: 40.9 (#113)

BenchmarkClaude 3.5 HaikuTrinity Large Thinking
LMArena Expert12081360
GPQA Diamond38.1%—
MMLU-Pro60.5%—
Confabulations36.7%—
Vectara Hallucination Rate—6.9%
GPQA (HELM)36.3%—
MMLU74.3%—

Multimodal Not comparable

Claude 3.5 Haiku: 26.8 (#117), Trinity Large Thinking: —

BenchmarkClaude 3.5 HaikuTrinity Large Thinking
LMArena Vision1092—
GeoBench34%—

Multilingual Trinity Large Thinking leads

Claude 3.5 Haiku: 40.0 (#218), Trinity Large Thinking: 46.2 (#160)

BenchmarkClaude 3.5 HaikuTrinity Large Thinking
LMArena Non-English12381325
LMArena Chinese12291373
LMArena French12641374
LMArena German12371356
LMArena Japanese11751311
LMArena Korean11731306
LMArena Russian12531337
LMArena Spanish12611357

Instruction Following Trinity Large Thinking leads

Claude 3.5 Haiku: 62.9 (#234), Trinity Large Thinking: 70.5 (#162)

BenchmarkClaude 3.5 HaikuTrinity Large Thinking
LMArena Instruction Following12411334
LiveBench Instruction Following61.9%—
IFEval79.2%—

Long Context Trinity Large Thinking leads

Claude 3.5 Haiku: 38.3 (#200), Trinity Large Thinking: 41.3 (#144)

BenchmarkClaude 3.5 HaikuTrinity Large Thinking
LMArena Longer Query12611355

Writing & Preference Trinity Large Thinking leads

Claude 3.5 Haiku: 42.7 (#234), Trinity Large Thinking: 53.8 (#158)

BenchmarkClaude 3.5 HaikuTrinity Large Thinking
LMArena Text12551340
LMArena Creative Writing12331320
LMArena Multi-Turn12651342
Short-Story Creative Writing73.5%—
EQ-Bench Creative Writing1146—
WildBench76%—
LiveBench Language35.4%—

Frequently asked questions

Is Claude 3.5 Haiku better than Trinity Large Thinking?

Trinity Large Thinking is the stronger model overall, scoring 38.6 to 29.2 on the Noometry Index.

Is Claude 3.5 Haiku or Trinity Large Thinking better for coding?

Trinity Large Thinking scores higher on coding benchmarks: 34.1 versus 32.9 in the Noometry coding category.

How many benchmarks do Claude 3.5 Haiku and Trinity Large Thinking share?

19 benchmarks have published results for both models. Claude 3.5 Haiku has 49 scored results on Noometry and Trinity Large Thinking has 24.

Related comparisons

Go deeper