Noometry

Model comparison

Granite 4.0 H Small vs Llama 3.1-70B

Granite 4.0 H Small is the stronger model overall, scoring 36.5 to 29.6 on the Noometry Index.

Last verified . 18 shared benchmarks.

Granite 4.0 H Small IBM

36.5

Rank #214 Confirmed

Llama 3.1-70B Meta

29.6

Rank #308 Confirmed

Summary

  • They share 18 benchmarks with published results for both. Granite 4.0 H Small scores higher in 7 categories and Llama 3.1-70B in 1 category; 6 gaps are clear of the uncertainty.
  • The widest gap is in math, where Granite 4.0 H Small leads 31.4 to 13.5.
  • The biggest single-benchmark swing is Omni-MATH: 29.6% for Granite 4.0 H Small and 21% for Llama 3.1-70B.

Side by side

Granite 4.0 H SmallLlama 3.1-70B
ProviderIBMMeta
Noometry Index36.529.6
Released—2024-07-23
WeightsOpenOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.40
Output $ / M tokens—$0.40
Results tracked1935

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Granite 4.0 H Small leads

Granite 4.0 H Small: 36.4 (#209), Llama 3.1-70B: 30.3 (#296)

BenchmarkGranite 4.0 H SmallLlama 3.1-70B
LMArena Coding12491260
WeirdML—9%
BigCodeBench Instruct—46.1%
BigCodeBench Complete—54.8%

Agentic & Tool Use Not comparable

Granite 4.0 H Small: —, Llama 3.1-70B: 25.1 (#112)

BenchmarkGranite 4.0 H SmallLlama 3.1-70B
TheAgentCompany—6.9%
BALROG—27.9%

Reasoning Granite 4.0 H Small leads

Granite 4.0 H Small: 24.4 (#163), Llama 3.1-70B: 21.6 (#220)

BenchmarkGranite 4.0 H SmallLlama 3.1-70B
LMArena Hard Prompts12401241
DTBench—60%
LMCA—14.8%
Epoch Capabilities Index—125.92

Math Granite 4.0 H Small leads

Granite 4.0 H Small: 31.4 (#223), Llama 3.1-70B: 13.5 (#304)

BenchmarkGranite 4.0 H SmallLlama 3.1-70B
Omni-MATH29.6%21%
LMArena Math12471252
OTIS Mock AIME 2024-2025—3.6%
MATH Level 5—36.7%

Knowledge Granite 4.0 H Small leads

Granite 4.0 H Small: 32.0 (#215), Llama 3.1-70B: 24.2 (#269)

BenchmarkGranite 4.0 H SmallLlama 3.1-70B
MMLU-Pro56.9%65.3%
GPQA (HELM)38.3%42.6%
LMArena Expert12511209
GPQA Diamond—44.2%
Vectara Hallucination Rate5.2%—
MMLU—80.1%

Multilingual Too close to call

Granite 4.0 H Small: 38.6 (#228), Llama 3.1-70B: 38.8 (#225)

BenchmarkGranite 4.0 H SmallLlama 3.1-70B
LMArena Non-English12161219
LMArena Chinese12491215
LMArena Russian12011234
LMArena Spanish12581253
LMArena French—1261
LMArena German—1222
LMArena Japanese—1132
LMArena Korean—1140

Instruction Following Granite 4.0 H Small leads

Granite 4.0 H Small: 68.7 (#183), Llama 3.1-70B: 65.3 (#223)

BenchmarkGranite 4.0 H SmallLlama 3.1-70B
IFEval89%82.1%
LMArena Instruction Following12221231

Long Context Too close to call

Granite 4.0 H Small: 37.7 (#213), Llama 3.1-70B: 37.6 (#214)

BenchmarkGranite 4.0 H SmallLlama 3.1-70B
LMArena Longer Query12421241

Writing & Preference Granite 4.0 H Small leads

Granite 4.0 H Small: 43.8 (#227), Llama 3.1-70B: 35.4 (#267)

BenchmarkGranite 4.0 H SmallLlama 3.1-70B
LMArena Text12411261
LMArena Creative Writing12111232
WildBench73.9%75.8%
LMArena Multi-Turn12421256
EQ-Bench Creative Writing—784

Frequently asked questions

Is Granite 4.0 H Small better than Llama 3.1-70B?

Granite 4.0 H Small is the stronger model overall, scoring 36.5 to 29.6 on the Noometry Index.

Is Granite 4.0 H Small or Llama 3.1-70B better for coding?

Granite 4.0 H Small scores higher on coding benchmarks: 36.4 versus 30.3 in the Noometry coding category.

How many benchmarks do Granite 4.0 H Small and Llama 3.1-70B share?

18 benchmarks have published results for both models. Granite 4.0 H Small has 19 scored results on Noometry and Llama 3.1-70B has 35.

Related comparisons

Go deeper