Noometry

Model comparison

Claude 3.5 Haiku vs DBRX

Claude 3.5 Haiku and DBRX score almost the same on the Noometry Index (29.2 vs 29.4), so choose on price, context window or the category you care about most.

Last verified . 19 shared benchmarks.

Claude 3.5 Haiku Anthropic

29.2

Rank #315 Confirmed

DBRX Databricks

29.4

Rank #311 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Claude 3.5 Haiku scores higher in 6 categories and DBRX in 2 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in multilingual, where Claude 3.5 Haiku leads 40.0 to 29.3.
  • The biggest single-benchmark swing is MATH Level 5: 46.4% for Claude 3.5 Haiku and 11.7% for DBRX.
  • DBRX has downloadable open weights; the other is API-only.

Side by side

Claude 3.5 HaikuDBRX
ProviderAnthropicDatabricks
Noometry Index29.229.4
Released2024-10-222024-03-27
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked4921

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

Claude 3.5 Haiku: 32.9 (#265), DBRX: 32.9 (#266)

BenchmarkClaude 3.5 HaikuDBRX
LMArena Coding12861132
Aider Polyglot28%—
SciCode27.4%—
WeirdML30.7%—
BigCodeBench Instruct46.1%—
LiveBench Coding51.4%—
BigCodeBench Complete59%—
CadEval32%—
HumanEval+—70.1%
MBPP+—55.8%

Agentic & Tool Use Not comparable

Claude 3.5 Haiku: 28.0 (#95), DBRX: —

BenchmarkClaude 3.5 HaikuDBRX
BALROG19.3%—

Reasoning DBRX leads

Claude 3.5 Haiku: 17.7 (#290), DBRX: 21.4 (#222)

BenchmarkClaude 3.5 HaikuDBRX
LMArena Hard Prompts12511113
CritPt0%—
LiveBench Reasoning28.1%—
DTBench56.7%—
LiveBench Data Analysis48.5%—
Epoch Capabilities Index127.15—
LiveBench43.5%—

Math DBRX leads

Claude 3.5 Haiku: 14.7 (#300), DBRX: 24.3 (#269)

BenchmarkClaude 3.5 HaikuDBRX
LMArena Math12441145
MATH Level 546.4%11.7%
OTIS Mock AIME 2024-20254.3%—
Omni-MATH22.4%—
LiveBench Math35.5%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge Claude 3.5 Haiku leads

Claude 3.5 Haiku: 18.7 (#281), DBRX: 14.9 (#294)

BenchmarkClaude 3.5 HaikuDBRX
GPQA Diamond38.1%32.9%
LMArena Expert12081076
MMLU-Pro60.5%—
Confabulations36.7%—
GPQA (HELM)36.3%—
MMLU74.3%—

Multimodal Not comparable

Claude 3.5 Haiku: 26.8 (#117), DBRX: —

BenchmarkClaude 3.5 HaikuDBRX
LMArena Vision1092—
GeoBench34%—

Multilingual Claude 3.5 Haiku leads

Claude 3.5 Haiku: 40.0 (#218), DBRX: 29.3 (#268)

BenchmarkClaude 3.5 HaikuDBRX
LMArena Non-English12381071
LMArena Chinese12291068
LMArena French12641096
LMArena German12371057
LMArena Japanese1175990
LMArena Korean1173993
LMArena Russian12531078
LMArena Spanish12611064

Instruction Following Claude 3.5 Haiku leads

Claude 3.5 Haiku: 62.9 (#234), DBRX: 57.5 (#270)

BenchmarkClaude 3.5 HaikuDBRX
LMArena Instruction Following12411112
LiveBench Instruction Following61.9%—
IFEval79.2%—

Long Context Claude 3.5 Haiku leads

Claude 3.5 Haiku: 38.3 (#200), DBRX: 33.7 (#258)

BenchmarkClaude 3.5 HaikuDBRX
LMArena Longer Query12611112

Writing & Preference Claude 3.5 Haiku leads

Claude 3.5 Haiku: 42.7 (#234), DBRX: 33.6 (#275)

BenchmarkClaude 3.5 HaikuDBRX
LMArena Text12551119
LMArena Creative Writing12331104
LMArena Multi-Turn12651111
Short-Story Creative Writing73.5%—
EQ-Bench Creative Writing1146—
WildBench76%—
LiveBench Language35.4%—

Frequently asked questions

Is Claude 3.5 Haiku better than DBRX?

Claude 3.5 Haiku and DBRX score almost the same on the Noometry Index (29.2 vs 29.4), so choose on price, context window or the category you care about most.

Is Claude 3.5 Haiku or DBRX better for coding?

They score almost the same on coding (32.9 vs 32.9); test both on your own repository before choosing.

How many benchmarks do Claude 3.5 Haiku and DBRX share?

19 benchmarks have published results for both models. Claude 3.5 Haiku has 49 scored results on Noometry and DBRX has 21.

Related comparisons

Go deeper