Noometry

Model comparison

Command R+ vs DeepSeek-V2 (MoE-236B, May 2024)

Command R+ has enough public results to be ranked (#257); DeepSeek-V2 (MoE-236B, May 2024) does not yet, so treat this comparison as directional.

Last verified . 4 shared benchmarks.

Command R+ Cohere

32.4

Rank #257 Confirmed

DeepSeek-V2 (MoE-236B, May 2024) DeepSeek

40.3

Unranked Sparse

Summary

  • They share 4 benchmarks with published results for both. Command R+ scores higher in 0 categories and DeepSeek-V2 (MoE-236B, May 2024) in 1 category; one gap is clear of the uncertainty.
  • The widest gap is in coding, where DeepSeek-V2 (MoE-236B, May 2024) leads 40.4 to 29.1.
  • The biggest single-benchmark swing is BigCodeBench Complete: 41.9% for Command R+ and 59.4% for DeepSeek-V2 (MoE-236B, May 2024).

Side by side

Command R+DeepSeek-V2 (MoE-236B, May 2024)
ProviderCohereDeepSeek
Noometry Index32.440.3
Released2024-08-302024-05-07
WeightsOpenOpen
Context window128K—
Max output4K—
Input $ / M tokens$2.50—
Output $ / M tokens$10—
Results tracked3410

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding DeepSeek-V2 (MoE-236B, May 2024) leads

Command R+: 29.1 (#309), DeepSeek-V2 (MoE-236B, May 2024): 40.4 (#139)

BenchmarkCommand R+DeepSeek-V2 (MoE-236B, May 2024)
BigCodeBench Instruct33.8%48.9%
BigCodeBench Complete41.9%59.4%
LiveBench Coding19.1%—
LMArena Coding1187—
HumanEval+56.7%—
MBPP+63.5%—

Reasoning Not comparable

Command R+: 9.2 (#344), DeepSeek-V2 (MoE-236B, May 2024): —

BenchmarkCommand R+DeepSeek-V2 (MoE-236B, May 2024)
Epoch Capabilities Index119.34124.77
SimpleBench17.4%—
LiveBench Reasoning24.8%—
LMArena Hard Prompts1186—
DTBench54.9%—
LiveBench Data Analysis38.1%—
LMCA5%—
BIG-Bench Hard—78.8%
HellaSwag—87.1%
LiveBench31.8%—
PIQA—83.9%
WinoGrande—86.3%

Math Not comparable

Command R+: 28.9 (#242), DeepSeek-V2 (MoE-236B, May 2024): —

BenchmarkCommand R+DeepSeek-V2 (MoE-236B, May 2024)
LiveBench Math21.3%—
LMArena Math1188—

Knowledge Not comparable

Command R+: 36.4 (#169), DeepSeek-V2 (MoE-236B, May 2024): —

BenchmarkCommand R+DeepSeek-V2 (MoE-236B, May 2024)
MMLU69.4%78.4%
Vectara Hallucination Rate6.9%—
LMArena Expert1174—
ARC (AI2) Challenge—92.2%
TriviaQA—80%

Multilingual Not comparable

Command R+: 38.6 (#227), DeepSeek-V2 (MoE-236B, May 2024): —

BenchmarkCommand R+DeepSeek-V2 (MoE-236B, May 2024)
LMArena Non-English1216—
LMArena Chinese1226—
LMArena French1209—
LMArena German1216—
LMArena Japanese1166—
LMArena Korean1138—
LMArena Russian1227—
LMArena Spanish1189—

Instruction Following Not comparable

Command R+: 60.0 (#254), DeepSeek-V2 (MoE-236B, May 2024): —

BenchmarkCommand R+DeepSeek-V2 (MoE-236B, May 2024)
LiveBench Instruction Following57.6%—
LMArena Instruction Following1197—

Long Context Not comparable

Command R+: 37.3 (#219), DeepSeek-V2 (MoE-236B, May 2024): —

BenchmarkCommand R+DeepSeek-V2 (MoE-236B, May 2024)
LMArena Longer Query1230—

Writing & Preference Not comparable

Command R+: 43.5 (#228), DeepSeek-V2 (MoE-236B, May 2024): —

BenchmarkCommand R+DeepSeek-V2 (MoE-236B, May 2024)
LMArena Text1229—
LMArena Creative Writing1235—
LMArena Multi-Turn1213—
LiveBench Language29.7%—

Frequently asked questions

Is Command R+ better than DeepSeek-V2 (MoE-236B, May 2024)?

Command R+ has enough public results to be ranked (#257); DeepSeek-V2 (MoE-236B, May 2024) does not yet, so treat this comparison as directional.

Is Command R+ or DeepSeek-V2 (MoE-236B, May 2024) better for coding?

DeepSeek-V2 (MoE-236B, May 2024) scores higher on coding benchmarks: 40.4 versus 29.1 in the Noometry coding category.

How many benchmarks do Command R+ and DeepSeek-V2 (MoE-236B, May 2024) share?

4 benchmarks have published results for both models. Command R+ has 34 scored results on Noometry and DeepSeek-V2 (MoE-236B, May 2024) has 10.

Related comparisons

Go deeper