Model comparison

Codestral vs Mistral Medium 3.5

Mistral Medium 3.5 is the stronger model overall, scoring 40.2 to 30.6 on the Noometry Index. Codestral costs 6.7× less per token, which makes it the better buy when Mistral Medium 3.5's lead doesn't matter for your workload.

Last verified . 1 shared benchmarks.

Codestral Mistral AI

30.6

Rank #290 Reported

Mistral Medium 3.5 Mistral AI

40.2

Rank #152 Confirmed

Summary

  • They share 1 benchmark with published results for both. Codestral scores higher in 1 category and Mistral Medium 3.5 in 1 category; 2 gaps are clear of the uncertainty.
  • The widest gap is in coding, where Mistral Medium 3.5 leads 36.0 to 27.3.
  • The biggest single-benchmark swing is Kagi LLM Benchmark: 32.5% for Codestral and 41.4% for Mistral Medium 3.5.
  • Codestral is cheaper at $0.30 / $0.90 per million input/output tokens, against $1.50 / $7.50 for Mistral Medium 3.5.
  • Mistral Medium 3.5 accepts more context: 262K tokens versus 256K.
  • Mistral Medium 3.5 has downloadable open weights; the other is API-only.

Side by side

Codestral and Mistral Medium 3.5 specifications
CodestralMistral Medium 3.5
ProviderMistral AIMistral AI
Noometry Index30.640.2
Released2024-05-29—
WeightsProprietaryOpen
Context window256K262K
Max output8K210K
Input $ / M tokens$0.30$1.50
Output $ / M tokens$0.90$7.50
Results tracked722

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Medium 3.5 leads

Codestral: 27.3 (#321), Mistral Medium 3.5: 36.0 (#213)

Coding benchmarks
BenchmarkCodestralMistral Medium 3.5
Aider Polyglot11.1%—
LMArena WebDev—1264
BigCodeBench Instruct41.8%—
LMArena Coding—1461
BigCodeBench Complete52.5%—
ALE-Bench137.78—
HumanEval+73.8%—
MBPP+61.9%—

Reasoning Codestral leads

Codestral: 19.8 (#251), Mistral Medium 3.5: 17.3 (#295)

Reasoning benchmarks
BenchmarkCodestralMistral Medium 3.5
Kagi LLM Benchmark32.5%41.4%
NYT Connections (extended)—12.9%
LMArena Hard Prompts—1436
Epoch Capabilities Index—141.35

Math Not comparable

Codestral: —, Mistral Medium 3.5: 39.1 (#113)

Math benchmarks
BenchmarkCodestralMistral Medium 3.5
LMArena Math—1431

Knowledge Not comparable

Codestral: —, Mistral Medium 3.5: 40.0 (#126)

Knowledge benchmarks
BenchmarkCodestralMistral Medium 3.5
LMArena Expert—1432

Multimodal Not comparable

Codestral: —, Mistral Medium 3.5: 38.3 (#65)

Multimodal benchmarks
BenchmarkCodestralMistral Medium 3.5
LMArena Vision—1223

Multilingual Not comparable

Codestral: —, Mistral Medium 3.5: 51.9 (#100)

Multilingual benchmarks
BenchmarkCodestralMistral Medium 3.5
LMArena Non-English—1404
LMArena Chinese—1442
LMArena French—1448
LMArena German—1451
LMArena Korean—1385
LMArena Russian—1395
LMArena Spanish—1409

Instruction Following Not comparable

Codestral: —, Mistral Medium 3.5: 74.6 (#90)

Instruction Following benchmarks
BenchmarkCodestralMistral Medium 3.5
LMArena Instruction Following—1415

Long Context Not comparable

Codestral: —, Mistral Medium 3.5: 43.2 (#103)

Long Context benchmarks
BenchmarkCodestralMistral Medium 3.5
LMArena Longer Query—1415

Writing & Preference Not comparable

Codestral: —, Mistral Medium 3.5: 58.5 (#117)

Writing & Preference benchmarks
BenchmarkCodestralMistral Medium 3.5
LMArena Text—1421
LMArena Creative Writing—1374
EQ-Bench 4—993
LMArena Multi-Turn—1423

Frequently asked questions

Is Codestral better than Mistral Medium 3.5?

Mistral Medium 3.5 is the stronger model overall, scoring 40.2 to 30.6 on the Noometry Index. Codestral costs 6.7× less per token, which makes it the better buy when Mistral Medium 3.5's lead doesn't matter for your workload.

Which is cheaper, Codestral or Mistral Medium 3.5?

Codestral is cheaper. It lists at $0.30 per million input tokens and $0.90 per million output tokens; Mistral Medium 3.5 lists at $1.50 and $7.50.

Is Codestral or Mistral Medium 3.5 better for coding?

Mistral Medium 3.5 scores higher on coding benchmarks: 36.0 versus 27.3 in the Noometry coding category.

Which has the bigger context window?

Mistral Medium 3.5 does, with 262K tokens against 256K.

How many benchmarks do Codestral and Mistral Medium 3.5 share?

1 benchmark has published results for both models. Codestral has 7 scored results on Noometry and Mistral Medium 3.5 has 22.

Related comparisons

Go deeper