Model comparison

Command R+ vs Mistral Medium 3.5

Mistral Medium 3.5 is the stronger model overall, scoring 40.2 to 32.4 on the Noometry Index.

Last verified . 17 shared benchmarks.

Command R+ Cohere

32.4

Rank #257 Confirmed

Mistral Medium 3.5 Mistral AI

40.2

Rank #152 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Command R+ scores higher in 0 categories and Mistral Medium 3.5 in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Mistral Medium 3.5 leads 58.5 to 43.5.
  • Mistral Medium 3.5 is cheaper at $1.50 / $7.50 per million input/output tokens, against $2.50 / $10 for Command R+.
  • Mistral Medium 3.5 accepts more context: 262K tokens versus 128K.

Side by side

Command R+ and Mistral Medium 3.5 specifications
Command R+Mistral Medium 3.5
ProviderCohereMistral AI
Noometry Index32.440.2
Released2024-08-30—
WeightsOpenOpen
Context window128K262K
Max output4K210K
Input $ / M tokens$2.50$1.50
Output $ / M tokens$10$7.50
Results tracked3422

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Medium 3.5 leads

Command R+: 29.1 (#309), Mistral Medium 3.5: 36.0 (#213)

Coding benchmarks
BenchmarkCommand R+Mistral Medium 3.5
LMArena Coding11871461
LMArena WebDev—1264
BigCodeBench Instruct33.8%—
LiveBench Coding19.1%—
BigCodeBench Complete41.9%—
HumanEval+56.7%—
MBPP+63.5%—

Reasoning Mistral Medium 3.5 leads

Command R+: 9.2 (#344), Mistral Medium 3.5: 17.3 (#295)

Reasoning benchmarks
BenchmarkCommand R+Mistral Medium 3.5
LMArena Hard Prompts11861436
Epoch Capabilities Index119.34141.35
SimpleBench17.4%—
Kagi LLM Benchmark—41.4%
NYT Connections (extended)—12.9%
LiveBench Reasoning24.8%—
DTBench54.9%—
LiveBench Data Analysis38.1%—
LMCA5%—
LiveBench31.8%—

Math Mistral Medium 3.5 leads

Command R+: 28.9 (#242), Mistral Medium 3.5: 39.1 (#113)

Math benchmarks
BenchmarkCommand R+Mistral Medium 3.5
LMArena Math11881431
LiveBench Math21.3%—

Knowledge Mistral Medium 3.5 leads

Command R+: 36.4 (#169), Mistral Medium 3.5: 40.0 (#126)

Knowledge benchmarks
BenchmarkCommand R+Mistral Medium 3.5
LMArena Expert11741432
Vectara Hallucination Rate6.9%—
MMLU69.4%—

Multimodal Not comparable

Command R+: —, Mistral Medium 3.5: 38.3 (#65)

Multimodal benchmarks
BenchmarkCommand R+Mistral Medium 3.5
LMArena Vision—1223

Multilingual Mistral Medium 3.5 leads

Command R+: 38.6 (#227), Mistral Medium 3.5: 51.9 (#100)

Multilingual benchmarks
BenchmarkCommand R+Mistral Medium 3.5
LMArena Non-English12161404
LMArena Chinese12261442
LMArena French12091448
LMArena German12161451
LMArena Korean11381385
LMArena Russian12271395
LMArena Spanish11891409
LMArena Japanese1166—

Instruction Following Mistral Medium 3.5 leads

Command R+: 60.0 (#254), Mistral Medium 3.5: 74.6 (#90)

Instruction Following benchmarks
BenchmarkCommand R+Mistral Medium 3.5
LMArena Instruction Following11971415
LiveBench Instruction Following57.6%—

Long Context Mistral Medium 3.5 leads

Command R+: 37.3 (#219), Mistral Medium 3.5: 43.2 (#103)

Long Context benchmarks
BenchmarkCommand R+Mistral Medium 3.5
LMArena Longer Query12301415

Writing & Preference Mistral Medium 3.5 leads

Command R+: 43.5 (#228), Mistral Medium 3.5: 58.5 (#117)

Writing & Preference benchmarks
BenchmarkCommand R+Mistral Medium 3.5
LMArena Text12291421
LMArena Creative Writing12351374
LMArena Multi-Turn12131423
EQ-Bench 4—993
LiveBench Language29.7%—

Frequently asked questions

Is Command R+ better than Mistral Medium 3.5?

Mistral Medium 3.5 is the stronger model overall, scoring 40.2 to 32.4 on the Noometry Index.

Which is cheaper, Command R+ or Mistral Medium 3.5?

Mistral Medium 3.5 is cheaper. It lists at $1.50 per million input tokens and $7.50 per million output tokens; Command R+ lists at $2.50 and $10.

Is Command R+ or Mistral Medium 3.5 better for coding?

Mistral Medium 3.5 scores higher on coding benchmarks: 36.0 versus 29.1 in the Noometry coding category.

Which has the bigger context window?

Mistral Medium 3.5 does, with 262K tokens against 128K.

How many benchmarks do Command R+ and Mistral Medium 3.5 share?

17 benchmarks have published results for both models. Command R+ has 34 scored results on Noometry and Mistral Medium 3.5 has 22.

Related comparisons

Go deeper