Noometry

Model comparison

Gemma 2 2b IT vs Llama-3.3-70B-Instruct

Gemma 2 2b IT is the stronger model overall, scoring 33.1 to 30.6 on the Noometry Index.

Last verified . 17 shared benchmarks.

Gemma 2 2b IT Google

33.1

Rank #248 Confirmed

Llama-3.3-70B-Instruct Meta

30.6

Rank #291 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Gemma 2 2b IT scores higher in 4 categories and Llama-3.3-70B-Instruct in 4 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Gemma 2 2b IT leads 32.6 to 15.3.

Side by side

Gemma 2 2b ITLlama-3.3-70B-Instruct
ProviderGoogleMeta
Noometry Index33.130.6
Released—2024-12-06
WeightsOpenOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.10
Output $ / M tokens—$0.32
Results tracked1743

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemma 2 2b IT leads

Gemma 2 2b IT: 32.3 (#273), Llama-3.3-70B-Instruct: 31.0 (#290)

BenchmarkGemma 2 2b ITLlama-3.3-70B-Instruct
LMArena Coding11121268
SciCode—26%
WeirdML—14.4%
BigCodeBench Instruct—46.9%
LiveBench Coding—36.6%
BigCodeBench Complete—57.5%

Agentic & Tool Use Not comparable

Gemma 2 2b IT: —, Llama-3.3-70B-Instruct: 25.8 (#105)

BenchmarkGemma 2 2b ITLlama-3.3-70B-Instruct
Berkeley Function Calling Leaderboard—31.9%
BALROG—23%

Reasoning Gemma 2 2b IT leads

Gemma 2 2b IT: 21.4 (#224), Llama-3.3-70B-Instruct: 14.1 (#327)

BenchmarkGemma 2 2b ITLlama-3.3-70B-Instruct
LMArena Hard Prompts11131257
SimpleBench—19.9%
CritPt—0%
LiveBench Reasoning—50.8%
DTBench—59.5%
LiveBench Data Analysis—49.5%
LMCA—17.5%
Epoch Capabilities Index—127.33
ForecastBench—58.6
LiveBench—50.2%

Math Gemma 2 2b IT leads

Gemma 2 2b IT: 32.6 (#212), Llama-3.3-70B-Instruct: 15.3 (#298)

BenchmarkGemma 2 2b ITLlama-3.3-70B-Instruct
LMArena Math11351267
OTIS Mock AIME 2024-2025—5.1%
LiveBench Math—42.2%
MATH Level 5—41.6%

Knowledge Too close to call

Gemma 2 2b IT: 29.9 (#231), Llama-3.3-70B-Instruct: 30.6 (#226)

BenchmarkGemma 2 2b ITLlama-3.3-70B-Instruct
LMArena Expert10961225
GPQA Diamond—47.4%
Confabulations—22.8%
Vectara Hallucination Rate—4.1%
MMLU—86.3%

Multilingual Llama-3.3-70B-Instruct leads

Gemma 2 2b IT: 32.3 (#257), Llama-3.3-70B-Instruct: 39.9 (#220)

BenchmarkGemma 2 2b ITLlama-3.3-70B-Instruct
LMArena Non-English11211236
LMArena Chinese11321217
LMArena French11571281
LMArena German11141251
LMArena Japanese10831150
LMArena Korean10551143
LMArena Russian11181252
LMArena Spanish11401270

Instruction Following Llama-3.3-70B-Instruct leads

Gemma 2 2b IT: 57.8 (#263), Llama-3.3-70B-Instruct: 71.1 (#157)

BenchmarkGemma 2 2b ITLlama-3.3-70B-Instruct
LMArena Instruction Following11181242
LiveBench Instruction Following—82.7%

Long Context Gemma 2 2b IT leads

Gemma 2 2b IT: 34.3 (#250), Llama-3.3-70B-Instruct: 26.4 (#295)

BenchmarkGemma 2 2b ITLlama-3.3-70B-Instruct
LMArena Longer Query11301256
Fiction.LiveBench—33.3%

Writing & Preference Llama-3.3-70B-Instruct leads

Gemma 2 2b IT: 36.5 (#263), Llama-3.3-70B-Instruct: 47.6 (#207)

BenchmarkGemma 2 2b ITLlama-3.3-70B-Instruct
LMArena Text11561274
LMArena Creative Writing11471250
LMArena Multi-Turn11181280
LiveBench Language—39.2%

Frequently asked questions

Is Gemma 2 2b IT better than Llama-3.3-70B-Instruct?

Gemma 2 2b IT is the stronger model overall, scoring 33.1 to 30.6 on the Noometry Index.

Is Gemma 2 2b IT or Llama-3.3-70B-Instruct better for coding?

Gemma 2 2b IT scores higher on coding benchmarks: 32.3 versus 31.0 in the Noometry coding category.

How many benchmarks do Gemma 2 2b IT and Llama-3.3-70B-Instruct share?

17 benchmarks have published results for both models. Gemma 2 2b IT has 17 scored results on Noometry and Llama-3.3-70B-Instruct has 43.

Related comparisons

Go deeper