Noometry

Model comparison

Gemma 1.1 7b IT vs Llama 3.3 Nemotron 49b Super v1

Llama 3.3 Nemotron 49b Super v1 is the stronger model overall, scoring 40.1 to 31.3 on the Noometry Index.

Last verified . 10 shared benchmarks.

Gemma 1.1 7b IT Google

31.3

Rank #277 Confirmed

Llama 3.3 Nemotron 49b Super v1 NVIDIA

40.1

Rank #154 Confirmed

Summary

  • They share 10 benchmarks with published results for both. Gemma 1.1 7b IT scores higher in 0 categories and Llama 3.3 Nemotron 49b Super v1 in 6 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Llama 3.3 Nemotron 49b Super v1 leads 50.8 to 30.4.

Side by side

Gemma 1.1 7b ITLlama 3.3 Nemotron 49b Super v1
ProviderGoogleNVIDIA
Noometry Index31.340.1
Released——
WeightsOpenOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked1910

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3.3 Nemotron 49b Super v1 leads

Gemma 1.1 7b IT: 31.5 (#284), Llama 3.3 Nemotron 49b Super v1: 37.9 (#186)

BenchmarkGemma 1.1 7b ITLlama 3.3 Nemotron 49b Super v1
LMArena Coding10841296
HumanEval+35.4%—
MBPP+45%—

Reasoning Llama 3.3 Nemotron 49b Super v1 leads

Gemma 1.1 7b IT: 20.5 (#238), Llama 3.3 Nemotron 49b Super v1: 26.2 (#135)

BenchmarkGemma 1.1 7b ITLlama 3.3 Nemotron 49b Super v1
LMArena Hard Prompts10711311

Math Not comparable

Gemma 1.1 7b IT: 32.0 (#220), Llama 3.3 Nemotron 49b Super v1: —

BenchmarkGemma 1.1 7b ITLlama 3.3 Nemotron 49b Super v1
LMArena Math1107—

Knowledge Not comparable

Gemma 1.1 7b IT: 28.3 (#247), Llama 3.3 Nemotron 49b Super v1: —

BenchmarkGemma 1.1 7b ITLlama 3.3 Nemotron 49b Super v1
LMArena Expert1039—

Multilingual Llama 3.3 Nemotron 49b Super v1 leads

Gemma 1.1 7b IT: 28.1 (#273), Llama 3.3 Nemotron 49b Super v1: 41.1 (#211)

BenchmarkGemma 1.1 7b ITLlama 3.3 Nemotron 49b Super v1
LMArena Non-English10521253
LMArena Chinese10611277
LMArena Russian10461269
LMArena French1065—
LMArena German1054—
LMArena Japanese971—
LMArena Korean988—
LMArena Spanish1049—

Instruction Following Llama 3.3 Nemotron 49b Super v1 leads

Gemma 1.1 7b IT: 54.0 (#283), Llama 3.3 Nemotron 49b Super v1: 68.3 (#189)

BenchmarkGemma 1.1 7b ITLlama 3.3 Nemotron 49b Super v1
LMArena Instruction Following10571293

Long Context Llama 3.3 Nemotron 49b Super v1 leads

Gemma 1.1 7b IT: 32.1 (#272), Llama 3.3 Nemotron 49b Super v1: 39.5 (#176)

BenchmarkGemma 1.1 7b ITLlama 3.3 Nemotron 49b Super v1
LMArena Longer Query10561299

Writing & Preference Llama 3.3 Nemotron 49b Super v1 leads

Gemma 1.1 7b IT: 30.4 (#288), Llama 3.3 Nemotron 49b Super v1: 50.8 (#179)

BenchmarkGemma 1.1 7b ITLlama 3.3 Nemotron 49b Super v1
LMArena Text10941308
LMArena Creative Writing10601288
LMArena Multi-Turn10401315

Frequently asked questions

Is Gemma 1.1 7b IT better than Llama 3.3 Nemotron 49b Super v1?

Llama 3.3 Nemotron 49b Super v1 is the stronger model overall, scoring 40.1 to 31.3 on the Noometry Index.

Is Gemma 1.1 7b IT or Llama 3.3 Nemotron 49b Super v1 better for coding?

Llama 3.3 Nemotron 49b Super v1 scores higher on coding benchmarks: 37.9 versus 31.5 in the Noometry coding category.

How many benchmarks do Gemma 1.1 7b IT and Llama 3.3 Nemotron 49b Super v1 share?

10 benchmarks have published results for both models. Gemma 1.1 7b IT has 19 scored results on Noometry and Llama 3.3 Nemotron 49b Super v1 has 10.

Related comparisons

Go deeper