Noometry

Model comparison

Granite 3.0 8b Instruct vs Llama 3.1-70B

Granite 3.0 8b Instruct is the stronger model overall, scoring 31.1 to 29.4 on the Noometry Index.

Last verified . 14 shared benchmarks.

Granite 3.0 8b Instruct IBM

31.1

Rank #266 Confirmed

Llama 3.1-70B Meta

29.4

Rank #299 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Granite 3.0 8b Instruct scores higher in 2 categories and Llama 3.1-70B in 6 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in math, where Granite 3.0 8b Instruct leads 30.3 to 12.7.
  • The biggest single-benchmark swing is BigCodeBench Complete: 35.4% for Granite 3.0 8b Instruct and 54.8% for Llama 3.1-70B.

Side by side

Granite 3.0 8b InstructLlama 3.1-70B
ProviderIBMMeta
Noometry Index31.129.4
Released—2024-07-23
WeightsOpenOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.40
Output $ / M tokens—$0.40
Results tracked1435

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

Granite 3.0 8b Instruct: 29.1 (#297), Llama 3.1-70B: 29.8 (#292)

BenchmarkGranite 3.0 8b InstructLlama 3.1-70B
BigCodeBench Instruct29.3%46.1%
LMArena Coding11131260
BigCodeBench Complete35.4%54.8%
WeirdML—9%

Agentic & Tool Use Not comparable

Granite 3.0 8b Instruct: —, Llama 3.1-70B: 24.9 (#104)

BenchmarkGranite 3.0 8b InstructLlama 3.1-70B
TheAgentCompany—6.9%
BALROG—27.9%

Reasoning Too close to call

Granite 3.0 8b Instruct: 20.0 (#227), Llama 3.1-70B: 20.8 (#215)

BenchmarkGranite 3.0 8b InstructLlama 3.1-70B
LMArena Hard Prompts10921241
DTBench—60%
LMCA—14.8%
Epoch Capabilities Index—125.91

Math Granite 3.0 8b Instruct leads

Granite 3.0 8b Instruct: 30.3 (#211), Llama 3.1-70B: 12.7 (#298)

BenchmarkGranite 3.0 8b InstructLlama 3.1-70B
LMArena Math11431252
OTIS Mock AIME 2024-2025—3.6%
Omni-MATH—21%
MATH Level 5—36.7%

Knowledge Granite 3.0 8b Instruct leads

Granite 3.0 8b Instruct: 29.1 (#233), Llama 3.1-70B: 24.1 (#264)

BenchmarkGranite 3.0 8b InstructLlama 3.1-70B
LMArena Expert10871209
GPQA Diamond—44.2%
MMLU-Pro—65.3%
GPQA (HELM)—42.6%
MMLU—80.1%

Multilingual Llama 3.1-70B leads

Granite 3.0 8b Instruct: 27.4 (#271), Llama 3.1-70B: 38.9 (#220)

BenchmarkGranite 3.0 8b InstructLlama 3.1-70B
LMArena Non-English10371219
LMArena Chinese10631215
LMArena Russian10601234
LMArena French—1260
LMArena German—1222
LMArena Japanese—1132
LMArena Korean—1140
LMArena Spanish—1253

Instruction Following Llama 3.1-70B leads

Granite 3.0 8b Instruct: 56.2 (#271), Llama 3.1-70B: 65.5 (#218)

BenchmarkGranite 3.0 8b InstructLlama 3.1-70B
LMArena Instruction Following10881231
IFEval—82.1%

Long Context Llama 3.1-70B leads

Granite 3.0 8b Instruct: 34.0 (#247), Llama 3.1-70B: 37.6 (#208)

BenchmarkGranite 3.0 8b InstructLlama 3.1-70B
LMArena Longer Query11221241

Writing & Preference Llama 3.1-70B leads

Granite 3.0 8b Instruct: 31.2 (#280), Llama 3.1-70B: 35.6 (#262)

BenchmarkGranite 3.0 8b InstructLlama 3.1-70B
LMArena Text10961261
LMArena Creative Writing10711232
LMArena Multi-Turn10631256
EQ-Bench Creative Writing—784
WildBench—75.8%

Frequently asked questions

Is Granite 3.0 8b Instruct better than Llama 3.1-70B?

Granite 3.0 8b Instruct is the stronger model overall, scoring 31.1 to 29.4 on the Noometry Index.

Is Granite 3.0 8b Instruct or Llama 3.1-70B better for coding?

They score almost the same on coding (29.1 vs 29.8); test both on your own repository before choosing.

How many benchmarks do Granite 3.0 8b Instruct and Llama 3.1-70B share?

14 benchmarks have published results for both models. Granite 3.0 8b Instruct has 14 scored results on Noometry and Llama 3.1-70B has 35.

Related comparisons

Go deeper