Noometry

Model comparison

Command R+ vs Granite 3.1 8b Instruct

Command R+ and Granite 3.1 8b Instruct score almost the same on the Noometry Index (32.4 vs 32.4), so choose on price, context window or the category you care about most.

Last verified . 12 shared benchmarks.

Command R+ Cohere

32.4

Rank #257 Confirmed

Granite 3.1 8b Instruct IBM

32.4

Rank #258 Confirmed

Summary

  • They share 12 benchmarks with published results for both. Command R+ scores higher in 5 categories and Granite 3.1 8b Instruct in 3 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Granite 3.1 8b Instruct leads 22.1 to 9.2.

Side by side

Command R+Granite 3.1 8b Instruct
ProviderCohereIBM
Noometry Index32.432.4
Released2024-08-30—
WeightsOpenOpen
Context window128K—
Max output4K—
Input $ / M tokens$2.50—
Output $ / M tokens$10—
Results tracked3413

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Granite 3.1 8b Instruct leads

Command R+: 29.1 (#309), Granite 3.1 8b Instruct: 34.5 (#233)

BenchmarkCommand R+Granite 3.1 8b Instruct
LMArena Coding11871186
BigCodeBench Instruct33.8%—
LiveBench Coding19.1%—
BigCodeBench Complete41.9%—
HumanEval+56.7%—
MBPP+63.5%—

Agentic & Tool Use Not comparable

Command R+: —, Granite 3.1 8b Instruct: 24.1 (#120)

BenchmarkCommand R+Granite 3.1 8b Instruct
Berkeley Function Calling Leaderboard—27.1%

Reasoning Granite 3.1 8b Instruct leads

Command R+: 9.2 (#344), Granite 3.1 8b Instruct: 22.1 (#207)

BenchmarkCommand R+Granite 3.1 8b Instruct
LMArena Hard Prompts11861145
SimpleBench17.4%—
LiveBench Reasoning24.8%—
DTBench54.9%—
LiveBench Data Analysis38.1%—
LMCA5%—
Epoch Capabilities Index119.34—
LiveBench31.8%—

Math Granite 3.1 8b Instruct leads

Command R+: 28.9 (#242), Granite 3.1 8b Instruct: 33.0 (#209)

BenchmarkCommand R+Granite 3.1 8b Instruct
LMArena Math11881152
LiveBench Math21.3%—

Knowledge Command R+ leads

Command R+: 36.4 (#169), Granite 3.1 8b Instruct: 31.1 (#220)

BenchmarkCommand R+Granite 3.1 8b Instruct
LMArena Expert11741142
Vectara Hallucination Rate6.9%—
MMLU69.4%—

Multilingual Command R+ leads

Command R+: 38.6 (#227), Granite 3.1 8b Instruct: 30.9 (#260)

BenchmarkCommand R+Granite 3.1 8b Instruct
LMArena Non-English12161099
LMArena Chinese12261145
LMArena Russian12271092
LMArena French1209—
LMArena German1216—
LMArena Japanese1166—
LMArena Korean1138—
LMArena Spanish1189—

Instruction Following Command R+ leads

Command R+: 60.0 (#254), Granite 3.1 8b Instruct: 58.6 (#259)

BenchmarkCommand R+Granite 3.1 8b Instruct
LMArena Instruction Following11971131
LiveBench Instruction Following57.6%—

Long Context Command R+ leads

Command R+: 37.3 (#219), Granite 3.1 8b Instruct: 35.2 (#241)

BenchmarkCommand R+Granite 3.1 8b Instruct
LMArena Longer Query12301162

Writing & Preference Command R+ leads

Command R+: 43.5 (#228), Granite 3.1 8b Instruct: 35.5 (#266)

BenchmarkCommand R+Granite 3.1 8b Instruct
LMArena Text12291150
LMArena Creative Writing12351129
LMArena Multi-Turn12131108
LiveBench Language29.7%—

Frequently asked questions

Is Command R+ better than Granite 3.1 8b Instruct?

Command R+ and Granite 3.1 8b Instruct score almost the same on the Noometry Index (32.4 vs 32.4), so choose on price, context window or the category you care about most.

Is Command R+ or Granite 3.1 8b Instruct better for coding?

Granite 3.1 8b Instruct scores higher on coding benchmarks: 34.5 versus 29.1 in the Noometry coding category.

How many benchmarks do Command R+ and Granite 3.1 8b Instruct share?

12 benchmarks have published results for both models. Command R+ has 34 scored results on Noometry and Granite 3.1 8b Instruct has 13.

Related comparisons

Go deeper