Noometry

Model comparison

Command R+ vs o1-mini

o1-mini is the stronger model overall, scoring 33.7 to 32.0 on the Noometry Index.

Last verified . 28 shared benchmarks.

Command R+ Cohere

32.0

Rank #251 Confirmed

o1-mini OpenAI

33.7

Rank #230 Confirmed

Summary

  • They share 28 benchmarks with published results for both. Command R+ scores higher in 1 category and o1-mini in 7 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where o1-mini leads 33.7 to 26.6.
  • The biggest single-benchmark swing is LiveBench Reasoning: 24.8% for Command R+ and 72.3% for o1-mini.
  • Command R+ has downloadable open weights; the other is API-only.

Side by side

Command R+o1-mini
ProviderCohereOpenAI
Noometry Index32.033.7
Released2024-08-302024-09-12
WeightsOpenProprietary
Context window128K—
Max output4K—
Input $ / M tokens$2.50—
Output $ / M tokens$10—
Results tracked3439

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o1-mini leads

Command R+: 28.6 (#304), o1-mini: 35.1 (#218)

BenchmarkCommand R+o1-mini
LiveBench Coding19.1%48%
LMArena Coding11871362
HumanEval+56.7%89%
MBPP+63.5%78.8%
Aider Polyglot—32.9%
WeirdML—36.3%
BigCodeBench Instruct33.8%—
BigCodeBench Complete41.9%—

Agentic & Tool Use Not comparable

Command R+: —, o1-mini: 24.3 (#112)

BenchmarkCommand R+o1-mini
Cybench—10%

Reasoning Too close to call

Command R+: 8.9 (#341), o1-mini: 8.9 (#340)

BenchmarkCommand R+o1-mini
SimpleBench17.4%18.1%
LiveBench Reasoning24.8%72.3%
LMArena Hard Prompts11861333
LiveBench Data Analysis38.1%57.9%
Epoch Capabilities Index119.28135.82
LiveBench31.8%57.8%
ARC-AGI-2—0.8%
ARC-AGI-1—14%
DTBench54.9%—
LMCA5%—

Math o1-mini leads

Command R+: 26.6 (#239), o1-mini: 33.7 (#168)

BenchmarkCommand R+o1-mini
LiveBench Math21.3%62%
LMArena Math11881358
OTIS Mock AIME 2024-2025—46.9%
MATH Level 5—89.2%
FrontierMath (Feb 2025 set)—1.7%

Knowledge Command R+ leads

Command R+: 35.9 (#165), o1-mini: 34.8 (#180)

BenchmarkCommand R+o1-mini
LMArena Expert11741316
GPQA Diamond—62.4%
Confabulations—18.6%
Vectara Hallucination Rate6.9%—
MMLU69.4%—

Multilingual o1-mini leads

Command R+: 38.8 (#222), o1-mini: 43.8 (#177)

BenchmarkCommand R+o1-mini
LMArena Non-English12161289
LMArena Chinese12261314
LMArena French12081292
LMArena German12161278
LMArena Japanese11661245
LMArena Korean11381223
LMArena Russian12271283
LMArena Spanish11891302

Instruction Following o1-mini leads

Command R+: 60.2 (#249), o1-mini: 67.0 (#200)

BenchmarkCommand R+o1-mini
LiveBench Instruction Following57.6%65.4%
LMArena Instruction Following11971304

Long Context o1-mini leads

Command R+: 37.3 (#214), o1-mini: 40.1 (#156)

BenchmarkCommand R+o1-mini
LMArena Longer Query12311321

Writing & Preference o1-mini leads

Command R+: 43.7 (#223), o1-mini: 48.6 (#196)

BenchmarkCommand R+o1-mini
LMArena Text12291317
LMArena Creative Writing12351244
LMArena Multi-Turn12131314
LiveBench Language29.7%40.9%
Short-Story Creative Writing—64.9%

Frequently asked questions

Is Command R+ better than o1-mini?

o1-mini is the stronger model overall, scoring 33.7 to 32.0 on the Noometry Index.

Is Command R+ or o1-mini better for coding?

o1-mini scores higher on coding benchmarks: 35.1 versus 28.6 in the Noometry coding category.

How many benchmarks do Command R+ and o1-mini share?

28 benchmarks have published results for both models. Command R+ has 34 scored results on Noometry and o1-mini has 39.

Related comparisons

Go deeper