Noometry

Model comparison

Granite 3.0 8b Instruct vs Nemotron-4 15B

Granite 3.0 8b Instruct has enough public results to be ranked (#266); Nemotron-4 15B does not yet, so treat this comparison as directional.

Last verified . 0 shared benchmarks.

Granite 3.0 8b Instruct IBM

31.1

Rank #266 Confirmed

Nemotron-4 15B NVIDIA

—

Unranked

Summary

  • Granite 3.0 8b Instruct has downloadable open weights; the other is API-only.

Side by side

Granite 3.0 8b InstructNemotron-4 15B
ProviderIBMNVIDIA
Noometry Index31.1—
Released—2024-02-26
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked148

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Granite 3.0 8b Instruct: 29.1 (#297), Nemotron-4 15B: —

BenchmarkGranite 3.0 8b InstructNemotron-4 15B
BigCodeBench Instruct29.3%—
LMArena Coding1113—
BigCodeBench Complete35.4%—

Reasoning Not comparable

Granite 3.0 8b Instruct: 20.0 (#227), Nemotron-4 15B: —

BenchmarkGranite 3.0 8b InstructNemotron-4 15B
LMArena Hard Prompts1092—
BIG-Bench Hard—58.7%
Epoch Capabilities Index—107.38
HellaSwag—82.4%
PIQA—82.4%
WinoGrande—78%

Math Not comparable

Granite 3.0 8b Instruct: 30.3 (#211), Nemotron-4 15B: —

BenchmarkGranite 3.0 8b InstructNemotron-4 15B
LMArena Math1143—
GSM8K—46%

Knowledge Not comparable

Granite 3.0 8b Instruct: 29.1 (#233), Nemotron-4 15B: —

BenchmarkGranite 3.0 8b InstructNemotron-4 15B
LMArena Expert1087—
ARC (AI2) Challenge—55.5%
MMLU—58.7%

Multilingual Not comparable

Granite 3.0 8b Instruct: 27.4 (#271), Nemotron-4 15B: —

BenchmarkGranite 3.0 8b InstructNemotron-4 15B
LMArena Non-English1037—
LMArena Chinese1063—
LMArena Russian1060—

Instruction Following Not comparable

Granite 3.0 8b Instruct: 56.2 (#271), Nemotron-4 15B: —

BenchmarkGranite 3.0 8b InstructNemotron-4 15B
LMArena Instruction Following1088—

Long Context Not comparable

Granite 3.0 8b Instruct: 34.0 (#247), Nemotron-4 15B: —

BenchmarkGranite 3.0 8b InstructNemotron-4 15B
LMArena Longer Query1122—

Writing & Preference Not comparable

Granite 3.0 8b Instruct: 31.2 (#280), Nemotron-4 15B: —

BenchmarkGranite 3.0 8b InstructNemotron-4 15B
LMArena Text1096—
LMArena Creative Writing1071—
LMArena Multi-Turn1063—

Frequently asked questions

Is Granite 3.0 8b Instruct better than Nemotron-4 15B?

Granite 3.0 8b Instruct has enough public results to be ranked (#266); Nemotron-4 15B does not yet, so treat this comparison as directional.

How many benchmarks do Granite 3.0 8b Instruct and Nemotron-4 15B share?

0 benchmarks have published results for both models. Granite 3.0 8b Instruct has 14 scored results on Noometry and Nemotron-4 15B has 8.

Related comparisons

Go deeper