Noometry

Model comparison

Granite 3.1 8b Instruct vs Nemotron-4 15B

Granite 3.1 8b Instruct has enough public results to be ranked (#256); Nemotron-4 15B does not yet, so treat this comparison as directional.

Last verified . 0 shared benchmarks.

Granite 3.1 8b Instruct IBM

31.8

Rank #256 Confirmed

Nemotron-4 15B NVIDIA

—

Unranked

Summary

  • Granite 3.1 8b Instruct has downloadable open weights; the other is API-only.

Side by side

Granite 3.1 8b InstructNemotron-4 15B
ProviderIBMNVIDIA
Noometry Index31.8—
Released—2024-02-26
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked138

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Granite 3.1 8b Instruct: 33.9 (#232), Nemotron-4 15B: —

BenchmarkGranite 3.1 8b InstructNemotron-4 15B
LMArena Coding1186—

Agentic & Tool Use Not comparable

Granite 3.1 8b Instruct: 23.9 (#113), Nemotron-4 15B: —

BenchmarkGranite 3.1 8b InstructNemotron-4 15B
Berkeley Function Calling Leaderboard27.1%—

Reasoning Not comparable

Granite 3.1 8b Instruct: 21.2 (#204), Nemotron-4 15B: —

BenchmarkGranite 3.1 8b InstructNemotron-4 15B
LMArena Hard Prompts1146—
BIG-Bench Hard—58.7%
Epoch Capabilities Index—107.38
HellaSwag—82.4%
PIQA—82.4%
WinoGrande—78%

Math Not comparable

Granite 3.1 8b Instruct: 30.4 (#210), Nemotron-4 15B: —

BenchmarkGranite 3.1 8b InstructNemotron-4 15B
LMArena Math1152—
GSM8K—46%

Knowledge Not comparable

Granite 3.1 8b Instruct: 30.6 (#218), Nemotron-4 15B: —

BenchmarkGranite 3.1 8b InstructNemotron-4 15B
LMArena Expert1142—
ARC (AI2) Challenge—55.5%
MMLU—58.7%

Multilingual Not comparable

Granite 3.1 8b Instruct: 31.1 (#255), Nemotron-4 15B: —

BenchmarkGranite 3.1 8b InstructNemotron-4 15B
LMArena Non-English1099—
LMArena Chinese1145—
LMArena Russian1092—

Instruction Following Not comparable

Granite 3.1 8b Instruct: 58.9 (#254), Nemotron-4 15B: —

BenchmarkGranite 3.1 8b InstructNemotron-4 15B
LMArena Instruction Following1131—

Long Context Not comparable

Granite 3.1 8b Instruct: 35.2 (#236), Nemotron-4 15B: —

BenchmarkGranite 3.1 8b InstructNemotron-4 15B
LMArena Longer Query1162—

Writing & Preference Not comparable

Granite 3.1 8b Instruct: 35.7 (#261), Nemotron-4 15B: —

BenchmarkGranite 3.1 8b InstructNemotron-4 15B
LMArena Text1150—
LMArena Creative Writing1129—
LMArena Multi-Turn1108—

Frequently asked questions

Is Granite 3.1 8b Instruct better than Nemotron-4 15B?

Granite 3.1 8b Instruct has enough public results to be ranked (#256); Nemotron-4 15B does not yet, so treat this comparison as directional.

How many benchmarks do Granite 3.1 8b Instruct and Nemotron-4 15B share?

0 benchmarks have published results for both models. Granite 3.1 8b Instruct has 13 scored results on Noometry and Nemotron-4 15B has 8.

Related comparisons

Go deeper