Noometry

Model comparison

Claude 2 vs Gemma 3n E4b IT

Gemma 3n E4b IT is the stronger model overall, scoring 37.3 to 25.0 on the Noometry Index.

Last verified . 0 shared benchmarks.

Claude 2 Anthropic

25.0

Rank #346 Reported

Gemma 3n E4b IT Google

37.3

Rank #206 Confirmed

Summary

  • The widest gap is in math, where Gemma 3n E4b IT leads 35.1 to 9.3.
  • Gemma 3n E4b IT has downloadable open weights; the other is API-only.

Side by side

Claude 2Gemma 3n E4b IT
ProviderAnthropicGoogle
Noometry Index25.037.3
Released2023-07-11—
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked818

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Claude 2: —, Gemma 3n E4b IT: 37.0 (#198)

BenchmarkClaude 2Gemma 3n E4b IT
LMArena Coding—1268
HumanEval+61.6%—

Reasoning Claude 2 leads

Claude 2: 21.7 (#216), Gemma 3n E4b IT: 19.9 (#247)

BenchmarkClaude 2Gemma 3n E4b IT
Kagi LLM Benchmark—31.5%
LMArena Hard Prompts—1284
DTBench51.9%—
Epoch Capabilities Index120.13—

Math Gemma 3n E4b IT leads

Claude 2: 9.3 (#320), Gemma 3n E4b IT: 35.1 (#188)

BenchmarkClaude 2Gemma 3n E4b IT
OTIS Mock AIME 2024-20252.5%—
LMArena Math—1251
MATH Level 511.7%—

Knowledge Gemma 3n E4b IT leads

Claude 2: 16.9 (#287), Gemma 3n E4b IT: 34.2 (#198)

BenchmarkClaude 2Gemma 3n E4b IT
GPQA Diamond34.7%—
LMArena Expert—1246
MMLU78.5%—
TriviaQA87.5%—

Multilingual Not comparable

Claude 2: —, Gemma 3n E4b IT: 43.4 (#183)

BenchmarkClaude 2Gemma 3n E4b IT
LMArena Non-English—1285
LMArena Chinese—1309
LMArena French—1330
LMArena German—1311
LMArena Japanese—1272
LMArena Korean—1259
LMArena Russian—1288
LMArena Spanish—1305

Instruction Following Not comparable

Claude 2: —, Gemma 3n E4b IT: 66.1 (#210)

BenchmarkClaude 2Gemma 3n E4b IT
LMArena Instruction Following—1255

Long Context Not comparable

Claude 2: —, Gemma 3n E4b IT: 38.7 (#191)

BenchmarkClaude 2Gemma 3n E4b IT
LMArena Longer Query—1276

Writing & Preference Not comparable

Claude 2: —, Gemma 3n E4b IT: 50.1 (#186)

BenchmarkClaude 2Gemma 3n E4b IT
LMArena Text—1306
LMArena Creative Writing—1287
LMArena Multi-Turn—1276

Frequently asked questions

Is Claude 2 better than Gemma 3n E4b IT?

Gemma 3n E4b IT is the stronger model overall, scoring 37.3 to 25.0 on the Noometry Index.

How many benchmarks do Claude 2 and Gemma 3n E4b IT share?

0 benchmarks have published results for both models. Claude 2 has 8 scored results on Noometry and Gemma 3n E4b IT has 18.

Related comparisons

Go deeper