Noometry

Model comparison

Command R+ vs Granite 3.0 8b Instruct

Command R+ and Granite 3.0 8b Instruct score almost the same on the Noometry Index (32.0 vs 31.1), so choose on price, context window or the category you care about most.

Last verified . 14 shared benchmarks.

Command R+ Cohere

32.0

Rank #251 Confirmed

Granite 3.0 8b Instruct IBM

31.1

Rank #266 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Command R+ scores higher in 5 categories and Granite 3.0 8b Instruct in 3 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Command R+ leads 43.7 to 31.2.
  • The biggest single-benchmark swing is BigCodeBench Complete: 41.9% for Command R+ and 35.4% for Granite 3.0 8b Instruct.

Side by side

Command R+Granite 3.0 8b Instruct
ProviderCohereIBM
Noometry Index32.031.1
Released2024-08-30—
WeightsOpenOpen
Context window128K—
Max output4K—
Input $ / M tokens$2.50—
Output $ / M tokens$10—
Results tracked3414

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

Command R+: 28.6 (#304), Granite 3.0 8b Instruct: 29.1 (#297)

BenchmarkCommand R+Granite 3.0 8b Instruct
BigCodeBench Instruct33.8%29.3%
LMArena Coding11871113
BigCodeBench Complete41.9%35.4%
LiveBench Coding19.1%—
HumanEval+56.7%—
MBPP+63.5%—

Reasoning Granite 3.0 8b Instruct leads

Command R+: 8.9 (#341), Granite 3.0 8b Instruct: 20.0 (#227)

BenchmarkCommand R+Granite 3.0 8b Instruct
LMArena Hard Prompts11861092
SimpleBench17.4%—
LiveBench Reasoning24.8%—
DTBench54.9%—
LiveBench Data Analysis38.1%—
LMCA5%—
Epoch Capabilities Index119.28—
LiveBench31.8%—

Math Granite 3.0 8b Instruct leads

Command R+: 26.6 (#239), Granite 3.0 8b Instruct: 30.3 (#211)

BenchmarkCommand R+Granite 3.0 8b Instruct
LMArena Math11881143
LiveBench Math21.3%—

Knowledge Command R+ leads

Command R+: 35.9 (#165), Granite 3.0 8b Instruct: 29.1 (#233)

BenchmarkCommand R+Granite 3.0 8b Instruct
LMArena Expert11741087
Vectara Hallucination Rate6.9%—
MMLU69.4%—

Multilingual Command R+ leads

Command R+: 38.8 (#222), Granite 3.0 8b Instruct: 27.4 (#271)

BenchmarkCommand R+Granite 3.0 8b Instruct
LMArena Non-English12161037
LMArena Chinese12261063
LMArena Russian12271060
LMArena French1208—
LMArena German1216—
LMArena Japanese1166—
LMArena Korean1138—
LMArena Spanish1189—

Instruction Following Command R+ leads

Command R+: 60.2 (#249), Granite 3.0 8b Instruct: 56.2 (#271)

BenchmarkCommand R+Granite 3.0 8b Instruct
LMArena Instruction Following11971088
LiveBench Instruction Following57.6%—

Long Context Command R+ leads

Command R+: 37.3 (#214), Granite 3.0 8b Instruct: 34.0 (#247)

BenchmarkCommand R+Granite 3.0 8b Instruct
LMArena Longer Query12311122

Writing & Preference Command R+ leads

Command R+: 43.7 (#223), Granite 3.0 8b Instruct: 31.2 (#280)

BenchmarkCommand R+Granite 3.0 8b Instruct
LMArena Text12291096
LMArena Creative Writing12351071
LMArena Multi-Turn12131063
LiveBench Language29.7%—

Frequently asked questions

Is Command R+ better than Granite 3.0 8b Instruct?

Command R+ and Granite 3.0 8b Instruct score almost the same on the Noometry Index (32.0 vs 31.1), so choose on price, context window or the category you care about most.

Is Command R+ or Granite 3.0 8b Instruct better for coding?

They score almost the same on coding (28.6 vs 29.1); test both on your own repository before choosing.

How many benchmarks do Command R+ and Granite 3.0 8b Instruct share?

14 benchmarks have published results for both models. Command R+ has 34 scored results on Noometry and Granite 3.0 8b Instruct has 14.

Related comparisons

Go deeper