Noometry

Model comparison

Command R vs Mixtral 8x7B

Command R is the stronger model overall, scoring 31.4 to 27.1 on the Noometry Index.

Last verified . 19 shared benchmarks.

Command R Cohere

31.4

Rank #272 Confirmed

Mixtral 8x7B Mistral AI

27.1

Rank #334 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Command R scores higher in 6 categories and Mixtral 8x7B in 2 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Command R leads 31.0 to 11.0.
  • Command R is cheaper at $0.15 / $0.60 per million input/output tokens, against $0.70 / $0.70 for Mixtral 8x7B.
  • Command R accepts more context: 128K tokens versus 32K.

Side by side

Command RMixtral 8x7B
ProviderCohereMistral AI
Noometry Index31.427.1
Released2024-08-302023-12-11
WeightsOpenOpen
Context window128K32K
Max output4K32K
Input $ / M tokens$0.15$0.70
Output $ / M tokens$0.60$0.70
Results tracked2938

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mixtral 8x7B leads

Command R: 29.3 (#306), Mixtral 8x7B: 32.8 (#269)

BenchmarkCommand RMixtral 8x7B
LMArena Coding11691126
BigCodeBench Instruct37.1%—
LiveBench Coding17.9%—
BigCodeBench Complete45.2%—
HumanEval+—39.6%
MBPP+—49.7%

Reasoning Mixtral 8x7B leads

Command R: 13.8 (#331), Mixtral 8x7B: 18.2 (#285)

BenchmarkCommand RMixtral 8x7B
LMArena Hard Prompts11641115
DTBench46.4%49.6%
LiveBench Reasoning21.9%—
LiveBench Data Analysis33.3%—
LMCA9.2%—
Adversarial NLI—55.2%
Epoch Capabilities Index—118.47
ForecastBench—56.3
HellaSwag—86.7%
LiveBench27.5%—
PIQA—83.6%
WinoGrande—77.2%

Math Command R leads

Command R: 28.0 (#246), Mixtral 8x7B: 18.8 (#289)

BenchmarkCommand RMixtral 8x7B
LMArena Math11551147
Omni-MATH—10.5%
LiveBench Math19.4%—
MATH Level 5—10%
GSM8K—74.4%

Knowledge Command R leads

Command R: 31.0 (#221), Mixtral 8x7B: 11.0 (#301)

BenchmarkCommand RMixtral 8x7B
LMArena Expert11381088
MMLU65.2%70.6%
GPQA Diamond—30.6%
MMLU-Pro—33.5%
GPQA (HELM)—29.6%
ARC (AI2) Challenge—87.3%
OpenBookQA—85.8%
TriviaQA—82.2%

Multilingual Command R leads

Command R: 35.7 (#245), Mixtral 8x7B: 29.6 (#266)

BenchmarkCommand RMixtral 8x7B
LMArena Non-English11741077
LMArena Chinese11821055
LMArena French11621166
LMArena German11761114
LMArena Japanese1143931
LMArena Korean1163968
LMArena Russian11741090
LMArena Spanish11511111

Instruction Following Command R leads

Command R: 58.1 (#261), Mixtral 8x7B: 51.0 (#297)

BenchmarkCommand RMixtral 8x7B
LMArena Instruction Following11671109
LiveBench Instruction Following55.6%—
IFEval—57.5%

Long Context Command R leads

Command R: 36.3 (#231), Mixtral 8x7B: 33.4 (#260)

BenchmarkCommand RMixtral 8x7B
LMArena Longer Query11981103

Writing & Preference Command R leads

Command R: 38.2 (#254), Mixtral 8x7B: 34.2 (#270)

BenchmarkCommand RMixtral 8x7B
LMArena Text11871132
LMArena Creative Writing11701109
LMArena Multi-Turn11631115
WildBench—67.3%
LiveBench Language16.7%—

Frequently asked questions

Is Command R better than Mixtral 8x7B?

Command R is the stronger model overall, scoring 31.4 to 27.1 on the Noometry Index.

Which is cheaper, Command R or Mixtral 8x7B?

Command R is cheaper. It lists at $0.15 per million input tokens and $0.60 per million output tokens; Mixtral 8x7B lists at $0.70 and $0.70.

Is Command R or Mixtral 8x7B better for coding?

Mixtral 8x7B scores higher on coding benchmarks: 32.8 versus 29.3 in the Noometry coding category.

Which has the bigger context window?

Command R does, with 128K tokens against 32K.

How many benchmarks do Command R and Mixtral 8x7B share?

19 benchmarks have published results for both models. Command R has 29 scored results on Noometry and Mixtral 8x7B has 38.

Related comparisons

Go deeper