Noometry

Model comparison

Gemma 3n E4b IT vs Llama-3.3-70B-Instruct

Gemma 3n E4b IT is the stronger model overall, scoring 37.3 to 30.6 on the Noometry Index.

Last verified . 17 shared benchmarks.

Gemma 3n E4b IT Google

37.3

Rank #206 Confirmed

Llama-3.3-70B-Instruct Meta

30.6

Rank #291 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Gemma 3n E4b IT scores higher in 7 categories and Llama-3.3-70B-Instruct in 1 category; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Gemma 3n E4b IT leads 35.1 to 15.3.

Side by side

Gemma 3n E4b ITLlama-3.3-70B-Instruct
ProviderGoogleMeta
Noometry Index37.330.6
Released—2024-12-06
WeightsOpenOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.10
Output $ / M tokens—$0.32
Results tracked1843

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemma 3n E4b IT leads

Gemma 3n E4b IT: 37.0 (#198), Llama-3.3-70B-Instruct: 31.0 (#290)

BenchmarkGemma 3n E4b ITLlama-3.3-70B-Instruct
LMArena Coding12681268
SciCode—26%
WeirdML—14.4%
BigCodeBench Instruct—46.9%
LiveBench Coding—36.6%
BigCodeBench Complete—57.5%

Agentic & Tool Use Not comparable

Gemma 3n E4b IT: —, Llama-3.3-70B-Instruct: 25.8 (#105)

BenchmarkGemma 3n E4b ITLlama-3.3-70B-Instruct
Berkeley Function Calling Leaderboard—31.9%
BALROG—23%

Reasoning Gemma 3n E4b IT leads

Gemma 3n E4b IT: 19.9 (#247), Llama-3.3-70B-Instruct: 14.1 (#327)

BenchmarkGemma 3n E4b ITLlama-3.3-70B-Instruct
LMArena Hard Prompts12841257
SimpleBench—19.9%
Kagi LLM Benchmark31.5%—
CritPt—0%
LiveBench Reasoning—50.8%
DTBench—59.5%
LiveBench Data Analysis—49.5%
LMCA—17.5%
Epoch Capabilities Index—127.33
ForecastBench—58.6
LiveBench—50.2%

Math Gemma 3n E4b IT leads

Gemma 3n E4b IT: 35.1 (#188), Llama-3.3-70B-Instruct: 15.3 (#298)

BenchmarkGemma 3n E4b ITLlama-3.3-70B-Instruct
LMArena Math12511267
OTIS Mock AIME 2024-2025—5.1%
LiveBench Math—42.2%
MATH Level 5—41.6%

Knowledge Gemma 3n E4b IT leads

Gemma 3n E4b IT: 34.2 (#198), Llama-3.3-70B-Instruct: 30.6 (#226)

BenchmarkGemma 3n E4b ITLlama-3.3-70B-Instruct
LMArena Expert12461225
GPQA Diamond—47.4%
Confabulations—22.8%
Vectara Hallucination Rate—4.1%
MMLU—86.3%

Multilingual Gemma 3n E4b IT leads

Gemma 3n E4b IT: 43.4 (#183), Llama-3.3-70B-Instruct: 39.9 (#220)

BenchmarkGemma 3n E4b ITLlama-3.3-70B-Instruct
LMArena Non-English12851236
LMArena Chinese13091217
LMArena French13301281
LMArena German13111251
LMArena Japanese12721150
LMArena Korean12591143
LMArena Russian12881252
LMArena Spanish13051270

Instruction Following Llama-3.3-70B-Instruct leads

Gemma 3n E4b IT: 66.1 (#210), Llama-3.3-70B-Instruct: 71.1 (#157)

BenchmarkGemma 3n E4b ITLlama-3.3-70B-Instruct
LMArena Instruction Following12551242
LiveBench Instruction Following—82.7%

Long Context Gemma 3n E4b IT leads

Gemma 3n E4b IT: 38.7 (#191), Llama-3.3-70B-Instruct: 26.4 (#295)

BenchmarkGemma 3n E4b ITLlama-3.3-70B-Instruct
LMArena Longer Query12761256
Fiction.LiveBench—33.3%

Writing & Preference Gemma 3n E4b IT leads

Gemma 3n E4b IT: 50.1 (#186), Llama-3.3-70B-Instruct: 47.6 (#207)

BenchmarkGemma 3n E4b ITLlama-3.3-70B-Instruct
LMArena Text13061274
LMArena Creative Writing12871250
LMArena Multi-Turn12761280
LiveBench Language—39.2%

Frequently asked questions

Is Gemma 3n E4b IT better than Llama-3.3-70B-Instruct?

Gemma 3n E4b IT is the stronger model overall, scoring 37.3 to 30.6 on the Noometry Index.

Is Gemma 3n E4b IT or Llama-3.3-70B-Instruct better for coding?

Gemma 3n E4b IT scores higher on coding benchmarks: 37.0 versus 31.0 in the Noometry coding category.

How many benchmarks do Gemma 3n E4b IT and Llama-3.3-70B-Instruct share?

17 benchmarks have published results for both models. Gemma 3n E4b IT has 18 scored results on Noometry and Llama-3.3-70B-Instruct has 43.

Related comparisons

Go deeper