Noometry

Model comparison

Command R+ vs Mixtral 8x22B

Command R+ is the stronger model overall, scoring 32.4 to 27.1 on the Noometry Index.

Last verified . 24 shared benchmarks.

Command R+ Cohere

32.4

Rank #257 Confirmed

Mixtral 8x22B Mistral AI

27.1

Rank #333 Confirmed

Summary

  • They share 24 benchmarks with published results for both. Command R+ scores higher in 7 categories and Mixtral 8x22B in 1 category; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Command R+ leads 36.4 to 15.1.
  • The biggest single-benchmark swing is BigCodeBench Complete: 41.9% for Command R+ and 50.2% for Mixtral 8x22B.
  • Mixtral 8x22B is cheaper at $2 / $6 per million input/output tokens, against $2.50 / $10 for Command R+.
  • Command R+ accepts more context: 128K tokens versus 64K.

Side by side

Command R+Mixtral 8x22B
ProviderCohereMistral AI
Noometry Index32.427.1
Released2024-08-302024-04-17
WeightsOpenOpen
Context window128K64K
Max output4K64K
Input $ / M tokens$2.50$2
Output $ / M tokens$10$6
Results tracked3434

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Command R+ leads

Command R+: 29.1 (#309), Mixtral 8x22B: 24.2 (#329)

BenchmarkCommand R+Mixtral 8x22B
BigCodeBench Instruct33.8%40.6%
LMArena Coding11871166
BigCodeBench Complete41.9%50.2%
HumanEval+56.7%72%
MBPP+63.5%64.3%
WeirdML—3.2%
LiveBench Coding19.1%—

Agentic & Tool Use Not comparable

Command R+: —, Mixtral 8x22B: 23.1 (#127)

BenchmarkCommand R+Mixtral 8x22B
Cybench—7.5%

Reasoning Mixtral 8x22B leads

Command R+: 9.2 (#344), Mixtral 8x22B: 19.9 (#248)

BenchmarkCommand R+Mixtral 8x22B
LMArena Hard Prompts11861150
DTBench54.9%55.1%
Epoch Capabilities Index119.34122.03
SimpleBench17.4%—
LiveBench Reasoning24.8%—
LiveBench Data Analysis38.1%—
LMCA5%—
ForecastBench—56.3
LiveBench31.8%—

Math Command R+ leads

Command R+: 28.9 (#242), Mixtral 8x22B: 22.9 (#275)

BenchmarkCommand R+Mixtral 8x22B
LMArena Math11881184
Omni-MATH—16.3%
LiveBench Math21.3%—
MATH Level 5—24.2%

Knowledge Command R+ leads

Command R+: 36.4 (#169), Mixtral 8x22B: 15.1 (#293)

BenchmarkCommand R+Mixtral 8x22B
LMArena Expert11741113
MMLU69.4%77.8%
GPQA Diamond—34.1%
MMLU-Pro—46%
Vectara Hallucination Rate6.9%—
GPQA (HELM)—33.4%

Multilingual Command R+ leads

Command R+: 38.6 (#227), Mixtral 8x22B: 32.8 (#255)

BenchmarkCommand R+Mixtral 8x22B
LMArena Non-English12161128
LMArena Chinese12261116
LMArena French12091166
LMArena German12161141
LMArena Japanese11661037
LMArena Korean11381057
LMArena Russian12271158
LMArena Spanish11891151

Instruction Following Command R+ leads

Command R+: 60.0 (#254), Mixtral 8x22B: 57.7 (#266)

BenchmarkCommand R+Mixtral 8x22B
LMArena Instruction Following11971147
LiveBench Instruction Following57.6%—
IFEval—72.4%

Long Context Command R+ leads

Command R+: 37.3 (#219), Mixtral 8x22B: 34.7 (#247)

BenchmarkCommand R+Mixtral 8x22B
LMArena Longer Query12301144

Writing & Preference Command R+ leads

Command R+: 43.5 (#228), Mixtral 8x22B: 36.9 (#262)

BenchmarkCommand R+Mixtral 8x22B
LMArena Text12291162
LMArena Creative Writing12351141
LMArena Multi-Turn12131130
WildBench—71.1%
LiveBench Language29.7%—

Frequently asked questions

Is Command R+ better than Mixtral 8x22B?

Command R+ is the stronger model overall, scoring 32.4 to 27.1 on the Noometry Index.

Which is cheaper, Command R+ or Mixtral 8x22B?

Mixtral 8x22B is cheaper. It lists at $2 per million input tokens and $6 per million output tokens; Command R+ lists at $2.50 and $10.

Is Command R+ or Mixtral 8x22B better for coding?

Command R+ scores higher on coding benchmarks: 29.1 versus 24.2 in the Noometry coding category.

Which has the bigger context window?

Command R+ does, with 128K tokens against 64K.

How many benchmarks do Command R+ and Mixtral 8x22B share?

24 benchmarks have published results for both models. Command R+ has 34 scored results on Noometry and Mixtral 8x22B has 34.

Related comparisons

Go deeper