Noometry

Model comparison

DBRX vs Mistral Medium 3.5

Mistral Medium 3.5 is the stronger model overall, scoring 40.2 to 29.4 on the Noometry Index.

Last verified . 16 shared benchmarks.

DBRX Databricks

29.4

Rank #311 Confirmed

Mistral Medium 3.5 Mistral AI

40.2

Rank #152 Confirmed

Summary

  • They share 16 benchmarks with published results for both. DBRX scores higher in 1 category and Mistral Medium 3.5 in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Mistral Medium 3.5 leads 40.0 to 14.9.

Side by side

DBRXMistral Medium 3.5
ProviderDatabricksMistral AI
Noometry Index29.440.2
Released2024-03-27—
WeightsOpenOpen
Context window—262K
Max output—210K
Input $ / M tokens—$1.50
Output $ / M tokens—$7.50
Results tracked2122

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Medium 3.5 leads

DBRX: 32.9 (#266), Mistral Medium 3.5: 36.0 (#213)

BenchmarkDBRXMistral Medium 3.5
LMArena Coding11321461
LMArena WebDev—1264
HumanEval+70.1%—
MBPP+55.8%—

Reasoning DBRX leads

DBRX: 21.4 (#222), Mistral Medium 3.5: 17.3 (#295)

BenchmarkDBRXMistral Medium 3.5
LMArena Hard Prompts11131436
Kagi LLM Benchmark—41.4%
NYT Connections (extended)—12.9%
Epoch Capabilities Index—141.35

Math Mistral Medium 3.5 leads

DBRX: 24.3 (#269), Mistral Medium 3.5: 39.1 (#113)

BenchmarkDBRXMistral Medium 3.5
LMArena Math11451431
MATH Level 511.7%—

Knowledge Mistral Medium 3.5 leads

DBRX: 14.9 (#294), Mistral Medium 3.5: 40.0 (#126)

BenchmarkDBRXMistral Medium 3.5
LMArena Expert10761432
GPQA Diamond32.9%—

Multimodal Not comparable

DBRX: —, Mistral Medium 3.5: 38.3 (#65)

BenchmarkDBRXMistral Medium 3.5
LMArena Vision—1223

Multilingual Mistral Medium 3.5 leads

DBRX: 29.3 (#268), Mistral Medium 3.5: 51.9 (#100)

BenchmarkDBRXMistral Medium 3.5
LMArena Non-English10711404
LMArena Chinese10681442
LMArena French10961448
LMArena German10571451
LMArena Korean9931385
LMArena Russian10781395
LMArena Spanish10641409
LMArena Japanese990—

Instruction Following Mistral Medium 3.5 leads

DBRX: 57.5 (#270), Mistral Medium 3.5: 74.6 (#90)

BenchmarkDBRXMistral Medium 3.5
LMArena Instruction Following11121415

Long Context Mistral Medium 3.5 leads

DBRX: 33.7 (#258), Mistral Medium 3.5: 43.2 (#103)

BenchmarkDBRXMistral Medium 3.5
LMArena Longer Query11121415

Writing & Preference Mistral Medium 3.5 leads

DBRX: 33.6 (#275), Mistral Medium 3.5: 58.5 (#117)

BenchmarkDBRXMistral Medium 3.5
LMArena Text11191421
LMArena Creative Writing11041374
LMArena Multi-Turn11111423
EQ-Bench 4—993

Frequently asked questions

Is DBRX better than Mistral Medium 3.5?

Mistral Medium 3.5 is the stronger model overall, scoring 40.2 to 29.4 on the Noometry Index.

Is DBRX or Mistral Medium 3.5 better for coding?

Mistral Medium 3.5 scores higher on coding benchmarks: 36.0 versus 32.9 in the Noometry coding category.

How many benchmarks do DBRX and Mistral Medium 3.5 share?

16 benchmarks have published results for both models. DBRX has 21 scored results on Noometry and Mistral Medium 3.5 has 22.

Related comparisons

Go deeper