Noometry

Model comparison

Claude 3.5 Sonnet vs DBRX

Claude 3.5 Sonnet is the stronger model overall, scoring 34.6 to 29.4 on the Noometry Index.

Last verified . 21 shared benchmarks.

Claude 3.5 Sonnet Anthropic

34.6

Rank #231 Confirmed

DBRX Databricks

29.4

Rank #311 Confirmed

Summary

  • They share 21 benchmarks with published results for both. Claude 3.5 Sonnet scores higher in 7 categories and DBRX in 1 category; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Claude 3.5 Sonnet leads 52.9 to 33.6.
  • The biggest single-benchmark swing is MATH Level 5: 56.9% for Claude 3.5 Sonnet and 11.7% for DBRX.
  • DBRX has downloadable open weights; the other is API-only.

Side by side

Claude 3.5 SonnetDBRX
ProviderAnthropicDatabricks
Noometry Index34.629.4
Released2024-06-202024-03-27
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked6021

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 39.0 (#165), DBRX: 32.9 (#266)

BenchmarkClaude 3.5 SonnetDBRX
LMArena Coding13421132
HumanEval+81.7%70.1%
MBPP+74.3%55.8%
Aider Polyglot51.6%—
GSO4.6%—
WeirdML40%—
BigCodeBench Instruct46.8%—
LiveBench Coding67.1%—
BigCodeBench Complete58.6%—
CadEval48%—

Agentic & Tool Use Not comparable

Claude 3.5 Sonnet: 32.3 (#67), DBRX: —

BenchmarkClaude 3.5 SonnetDBRX
TheAgentCompany24%—
Cybench17.5%—
BALROG32.6%—
METR Time Horizons45.2%—

Reasoning Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 23.1 (#183), DBRX: 21.4 (#222)

BenchmarkClaude 3.5 SonnetDBRX
LMArena Hard Prompts13051113
SimpleBench41.4%—
EnigmaEval0.9%—
LiveBench Reasoning56.7%—
DTBench67.8%—
LiveBench Data Analysis55%—
Epoch Capabilities Index133.55—
ForecastBench60.7—
LiveBench59%—

Math DBRX leads

Claude 3.5 Sonnet: 19.2 (#288), DBRX: 24.3 (#269)

BenchmarkClaude 3.5 SonnetDBRX
LMArena Math13071145
MATH Level 556.9%11.7%
OTIS Mock AIME 2024-20258.5%—
Omni-MATH27.6%—
LiveBench Math52.3%—
FrontierMath (Feb 2025 set)2.1%—
FrontierMath Tier 4 (v1)0%—

Knowledge Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 28.6 (#245), DBRX: 14.9 (#294)

BenchmarkClaude 3.5 SonnetDBRX
GPQA Diamond55.3%32.9%
LMArena Expert12651076
Humanity's Last Exam4.1%—
MMLU-Pro77.7%—
Confabulations19.9%—
GPQA (HELM)56.5%—
MMLU87.3%—

Multimodal Not comparable

Claude 3.5 Sonnet: 26.5 (#120), DBRX: —

BenchmarkClaude 3.5 SonnetDBRX
LMArena Vision1125—
Video-MME60%—
GeoBench62%—
VPCT33%—

Multilingual Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 43.2 (#185), DBRX: 29.3 (#268)

BenchmarkClaude 3.5 SonnetDBRX
LMArena Non-English12831071
LMArena Chinese12721068
LMArena French13051096
LMArena German12971057
LMArena Japanese1234990
LMArena Korean1200993
LMArena Russian13061078
LMArena Spanish12901064

Instruction Following Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 68.8 (#182), DBRX: 57.5 (#270)

BenchmarkClaude 3.5 SonnetDBRX
LMArena Instruction Following12971112
LiveBench Instruction Following69.3%—
IFEval85.5%—

Long Context Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 39.9 (#167), DBRX: 33.7 (#258)

BenchmarkClaude 3.5 SonnetDBRX
LMArena Longer Query13111112

Writing & Preference Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 52.9 (#164), DBRX: 33.6 (#275)

BenchmarkClaude 3.5 SonnetDBRX
LMArena Text12981119
LMArena Creative Writing12921104
LMArena Multi-Turn13261111
Short-Story Creative Writing80.3%—
EQ-Bench Creative Writing1451—
WildBench79.2%—
LiveBench Language53.8%—

Frequently asked questions

Is Claude 3.5 Sonnet better than DBRX?

Claude 3.5 Sonnet is the stronger model overall, scoring 34.6 to 29.4 on the Noometry Index.

Is Claude 3.5 Sonnet or DBRX better for coding?

Claude 3.5 Sonnet scores higher on coding benchmarks: 39.0 versus 32.9 in the Noometry coding category.

How many benchmarks do Claude 3.5 Sonnet and DBRX share?

21 benchmarks have published results for both models. Claude 3.5 Sonnet has 60 scored results on Noometry and DBRX has 21.

Related comparisons

Go deeper