Noometry

Model comparison

Command R+ vs Grok-2 (Dec 2024)

Grok-2 (Dec 2024) is the stronger model overall, scoring 33.7 to 32.4 on the Noometry Index.

Last verified . 27 shared benchmarks.

Command R+ Cohere

32.4

Rank #257 Confirmed

Grok-2 (Dec 2024) xAI

33.7

Rank #239 Confirmed

Summary

  • They share 27 benchmarks with published results for both. Command R+ scores higher in 2 categories and Grok-2 (Dec 2024) in 6 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Command R+ leads 28.9 to 20.8.
  • The biggest single-benchmark swing is LiveBench Math: 21.3% for Command R+ and 54.9% for Grok-2 (Dec 2024).
  • Command R+ has downloadable open weights; the other is API-only.

Side by side

Command R+Grok-2 (Dec 2024)
ProviderCoherexAI
Noometry Index32.433.7
Released2024-08-302024-08-13
WeightsOpenProprietary
Context window128K—
Max output4K—
Input $ / M tokens$2.50—
Output $ / M tokens$10—
Results tracked3434

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok-2 (Dec 2024) leads

Command R+: 29.1 (#309), Grok-2 (Dec 2024): 33.3 (#258)

BenchmarkCommand R+Grok-2 (Dec 2024)
LiveBench Coding19.1%46.4%
LMArena Coding11871287
WeirdML—22.2%
BigCodeBench Instruct33.8%—
BigCodeBench Complete41.9%—
HumanEval+56.7%—
MBPP+63.5%—

Reasoning Grok-2 (Dec 2024) leads

Command R+: 9.2 (#344), Grok-2 (Dec 2024): 16.9 (#299)

BenchmarkCommand R+Grok-2 (Dec 2024)
SimpleBench17.4%22.7%
LiveBench Reasoning24.8%54.8%
LMArena Hard Prompts11861272
DTBench54.9%65.2%
LiveBench Data Analysis38.1%54.5%
Epoch Capabilities Index119.34130.48
LiveBench31.8%54.3%
LMCA5%—

Math Command R+ leads

Command R+: 28.9 (#242), Grok-2 (Dec 2024): 20.8 (#284)

BenchmarkCommand R+Grok-2 (Dec 2024)
LiveBench Math21.3%54.9%
LMArena Math11881283
OTIS Mock AIME 2024-2025—11.5%
MATH Level 5—63.5%
FrontierMath (Feb 2025 set)—0.7%

Knowledge Command R+ leads

Command R+: 36.4 (#169), Grok-2 (Dec 2024): 29.8 (#233)

BenchmarkCommand R+Grok-2 (Dec 2024)
LMArena Expert11741254
GPQA Diamond—53.8%
Confabulations—20.1%
Vectara Hallucination Rate6.9%—
MMLU69.4%—

Multilingual Grok-2 (Dec 2024) leads

Command R+: 38.6 (#227), Grok-2 (Dec 2024): 43.1 (#188)

BenchmarkCommand R+Grok-2 (Dec 2024)
LMArena Non-English12161282
LMArena Chinese12261289
LMArena French12091318
LMArena German12161287
LMArena Japanese11661244
LMArena Korean11381237
LMArena Russian12271286
LMArena Spanish11891281

Instruction Following Grok-2 (Dec 2024) leads

Command R+: 60.0 (#254), Grok-2 (Dec 2024): 66.9 (#202)

BenchmarkCommand R+Grok-2 (Dec 2024)
LiveBench Instruction Following57.6%69.6%
LMArena Instruction Following11971270

Long Context Grok-2 (Dec 2024) leads

Command R+: 37.3 (#219), Grok-2 (Dec 2024): 38.8 (#190)

BenchmarkCommand R+Grok-2 (Dec 2024)
LMArena Longer Query12301276

Writing & Preference Grok-2 (Dec 2024) leads

Command R+: 43.5 (#228), Grok-2 (Dec 2024): 48.6 (#198)

BenchmarkCommand R+Grok-2 (Dec 2024)
LMArena Text12291305
LMArena Creative Writing12351284
LMArena Multi-Turn12131290
LiveBench Language29.7%45.6%
Short-Story Creative Writing—63.6%

Frequently asked questions

Is Command R+ better than Grok-2 (Dec 2024)?

Grok-2 (Dec 2024) is the stronger model overall, scoring 33.7 to 32.4 on the Noometry Index.

Is Command R+ or Grok-2 (Dec 2024) better for coding?

Grok-2 (Dec 2024) scores higher on coding benchmarks: 33.3 versus 29.1 in the Noometry coding category.

How many benchmarks do Command R+ and Grok-2 (Dec 2024) share?

27 benchmarks have published results for both models. Command R+ has 34 scored results on Noometry and Grok-2 (Dec 2024) has 34.

Related comparisons

Go deeper