Noometry

Model comparison

Gemma 2 9B vs Llama 3.1 Nemotron 70b Instruct

Llama 3.1 Nemotron 70b Instruct is the stronger model overall, scoring 37.6 to 25.9 on the Noometry Index.

Last verified . 14 shared benchmarks.

Gemma 2 9B Google

25.9

Rank #341 Confirmed

Llama 3.1 Nemotron 70b Instruct NVIDIA

37.6

Rank #203 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Gemma 2 9B scores higher in 0 categories and Llama 3.1 Nemotron 70b Instruct in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Llama 3.1 Nemotron 70b Instruct leads 35.5 to 9.9.
  • The biggest single-benchmark swing is BigCodeBench Complete: 40.6% for Gemma 2 9B and 48.2% for Llama 3.1 Nemotron 70b Instruct.

Side by side

Gemma 2 9BLlama 3.1 Nemotron 70b Instruct
ProviderGoogleNVIDIA
Noometry Index25.937.6
Released2024-06-242024-12-18
WeightsOpenOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3514

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3.1 Nemotron 70b Instruct leads

Gemma 2 9B: 29.4 (#304), Llama 3.1 Nemotron 70b Instruct: 35.9 (#216)

BenchmarkGemma 2 9BLlama 3.1 Nemotron 70b Instruct
BigCodeBench Instruct34.7%38.7%
LMArena Coding11731272
BigCodeBench Complete40.6%48.2%
LiveBench Coding22.5%—

Reasoning Llama 3.1 Nemotron 70b Instruct leads

Gemma 2 9B: 15.9 (#309), Llama 3.1 Nemotron 70b Instruct: 25.0 (#152)

BenchmarkGemma 2 9BLlama 3.1 Nemotron 70b Instruct
LMArena Hard Prompts11711266
LiveBench Reasoning15.2%—
LiveBench Data Analysis36.4%—
Epoch Capabilities Index119.83—
LiveBench28.7%—
PIQA83.7%—

Math Llama 3.1 Nemotron 70b Instruct leads

Gemma 2 9B: 9.9 (#318), Llama 3.1 Nemotron 70b Instruct: 35.5 (#182)

BenchmarkGemma 2 9BLlama 3.1 Nemotron 70b Instruct
LMArena Math11831271
OTIS Mock AIME 2024-20250.6%—
LiveBench Math19.8%—
MATH Level 521%—
GSM8K84.9%—

Knowledge Llama 3.1 Nemotron 70b Instruct leads

Gemma 2 9B: 9.7 (#305), Llama 3.1 Nemotron 70b Instruct: 34.1 (#199)

BenchmarkGemma 2 9BLlama 3.1 Nemotron 70b Instruct
LMArena Expert11471242
GPQA Diamond27.5%—
BoolQ85.7%—
MMLU72.1%—

Multilingual Llama 3.1 Nemotron 70b Instruct leads

Gemma 2 9B: 36.6 (#238), Llama 3.1 Nemotron 70b Instruct: 40.5 (#217)

BenchmarkGemma 2 9BLlama 3.1 Nemotron 70b Instruct
LMArena Non-English11881245
LMArena Chinese11851263
LMArena Russian12001227
LMArena French1190—
LMArena German1186—
LMArena Japanese1144—
LMArena Korean1137—
LMArena Spanish1200—

Instruction Following Llama 3.1 Nemotron 70b Instruct leads

Gemma 2 9B: 57.6 (#269), Llama 3.1 Nemotron 70b Instruct: 65.9 (#213)

BenchmarkGemma 2 9BLlama 3.1 Nemotron 70b Instruct
LMArena Instruction Following11781252
LiveBench Instruction Following52.6%—

Long Context Llama 3.1 Nemotron 70b Instruct leads

Gemma 2 9B: 36.3 (#233), Llama 3.1 Nemotron 70b Instruct: 37.6 (#215)

BenchmarkGemma 2 9BLlama 3.1 Nemotron 70b Instruct
LMArena Longer Query11971238

Writing & Preference Llama 3.1 Nemotron 70b Instruct leads

Gemma 2 9B: 32.1 (#281), Llama 3.1 Nemotron 70b Instruct: 48.4 (#203)

BenchmarkGemma 2 9BLlama 3.1 Nemotron 70b Instruct
LMArena Text12071283
LMArena Creative Writing12061269
LMArena Multi-Turn11931275
EQ-Bench Creative Writing841—
LiveBench Language25.5%—

Frequently asked questions

Is Gemma 2 9B better than Llama 3.1 Nemotron 70b Instruct?

Llama 3.1 Nemotron 70b Instruct is the stronger model overall, scoring 37.6 to 25.9 on the Noometry Index.

Is Gemma 2 9B or Llama 3.1 Nemotron 70b Instruct better for coding?

Llama 3.1 Nemotron 70b Instruct scores higher on coding benchmarks: 35.9 versus 29.4 in the Noometry coding category.

How many benchmarks do Gemma 2 9B and Llama 3.1 Nemotron 70b Instruct share?

14 benchmarks have published results for both models. Gemma 2 9B has 35 scored results on Noometry and Llama 3.1 Nemotron 70b Instruct has 14.

Related comparisons

Go deeper