Noometry

Model comparison

Command R vs Mixtral 8x22B

Command R is the stronger model overall, scoring 31.4 to 27.1 on the Noometry Index.

Last verified . 21 shared benchmarks.

Command R Cohere

31.4

Rank #272 Confirmed

Mixtral 8x22B Mistral AI

27.1

Rank #333 Confirmed

Summary

  • They share 21 benchmarks with published results for both. Command R scores higher in 7 categories and Mixtral 8x22B in 1 category; 7 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Command R leads 31.0 to 15.1.
  • The biggest single-benchmark swing is DTBench: 46.4% for Command R and 55.1% for Mixtral 8x22B.
  • Command R is cheaper at $0.15 / $0.60 per million input/output tokens, against $2 / $6 for Mixtral 8x22B.
  • Command R accepts more context: 128K tokens versus 64K.

Side by side

Command RMixtral 8x22B
ProviderCohereMistral AI
Noometry Index31.427.1
Released2024-08-302024-04-17
WeightsOpenOpen
Context window128K64K
Max output4K64K
Input $ / M tokens$0.15$2
Output $ / M tokens$0.60$6
Results tracked2934

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Command R leads

Command R: 29.3 (#306), Mixtral 8x22B: 24.2 (#329)

BenchmarkCommand RMixtral 8x22B
BigCodeBench Instruct37.1%40.6%
LMArena Coding11691166
BigCodeBench Complete45.2%50.2%
WeirdML—3.2%
LiveBench Coding17.9%—
HumanEval+—72%
MBPP+—64.3%

Agentic & Tool Use Not comparable

Command R: —, Mixtral 8x22B: 23.1 (#127)

BenchmarkCommand RMixtral 8x22B
Cybench—7.5%

Reasoning Mixtral 8x22B leads

Command R: 13.8 (#331), Mixtral 8x22B: 19.9 (#248)

BenchmarkCommand RMixtral 8x22B
LMArena Hard Prompts11641150
DTBench46.4%55.1%
LiveBench Reasoning21.9%—
LiveBench Data Analysis33.3%—
LMCA9.2%—
Epoch Capabilities Index—122.03
ForecastBench—56.3
LiveBench27.5%—

Math Command R leads

Command R: 28.0 (#246), Mixtral 8x22B: 22.9 (#275)

BenchmarkCommand RMixtral 8x22B
LMArena Math11551184
Omni-MATH—16.3%
LiveBench Math19.4%—
MATH Level 5—24.2%

Knowledge Command R leads

Command R: 31.0 (#221), Mixtral 8x22B: 15.1 (#293)

BenchmarkCommand RMixtral 8x22B
LMArena Expert11381113
MMLU65.2%77.8%
GPQA Diamond—34.1%
MMLU-Pro—46%
GPQA (HELM)—33.4%

Multilingual Command R leads

Command R: 35.7 (#245), Mixtral 8x22B: 32.8 (#255)

BenchmarkCommand RMixtral 8x22B
LMArena Non-English11741128
LMArena Chinese11821116
LMArena French11621166
LMArena German11761141
LMArena Japanese11431037
LMArena Korean11631057
LMArena Russian11741158
LMArena Spanish11511151

Instruction Following Too close to call

Command R: 58.1 (#261), Mixtral 8x22B: 57.7 (#266)

BenchmarkCommand RMixtral 8x22B
LMArena Instruction Following11671147
LiveBench Instruction Following55.6%—
IFEval—72.4%

Long Context Command R leads

Command R: 36.3 (#231), Mixtral 8x22B: 34.7 (#247)

BenchmarkCommand RMixtral 8x22B
LMArena Longer Query11981144

Writing & Preference Command R leads

Command R: 38.2 (#254), Mixtral 8x22B: 36.9 (#262)

BenchmarkCommand RMixtral 8x22B
LMArena Text11871162
LMArena Creative Writing11701141
LMArena Multi-Turn11631130
WildBench—71.1%
LiveBench Language16.7%—

Frequently asked questions

Is Command R better than Mixtral 8x22B?

Command R is the stronger model overall, scoring 31.4 to 27.1 on the Noometry Index.

Which is cheaper, Command R or Mixtral 8x22B?

Command R is cheaper. It lists at $0.15 per million input tokens and $0.60 per million output tokens; Mixtral 8x22B lists at $2 and $6.

Is Command R or Mixtral 8x22B better for coding?

Command R scores higher on coding benchmarks: 29.3 versus 24.2 in the Noometry coding category.

Which has the bigger context window?

Command R does, with 128K tokens against 64K.

How many benchmarks do Command R and Mixtral 8x22B share?

21 benchmarks have published results for both models. Command R has 29 scored results on Noometry and Mixtral 8x22B has 34.

Related comparisons

Go deeper