Noometry

Model comparison

Claude 2.1 vs Gemma 3n E4b IT

Gemma 3n E4b IT is the stronger model overall, scoring 37.3 to 25.2 on the Noometry Index.

Last verified . 0 shared benchmarks.

Claude 2.1 Anthropic

25.2

Rank #345 Reported

Gemma 3n E4b IT Google

37.3

Rank #206 Confirmed

Summary

  • The widest gap is in math, where Gemma 3n E4b IT leads 35.1 to 10.2.
  • Gemma 3n E4b IT has downloadable open weights; the other is API-only.

Side by side

Claude 2.1Gemma 3n E4b IT
ProviderAnthropicGoogle
Noometry Index25.237.3
Released2023-11-21—
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked718

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemma 3n E4b IT leads

Claude 2.1: 26.2 (#327), Gemma 3n E4b IT: 37.0 (#198)

BenchmarkClaude 2.1Gemma 3n E4b IT
WeirdML7.1%—
LMArena Coding—1268

Reasoning Claude 2.1 leads

Claude 2.1: 21.4 (#221), Gemma 3n E4b IT: 19.9 (#247)

BenchmarkClaude 2.1Gemma 3n E4b IT
Kagi LLM Benchmark—31.5%
LMArena Hard Prompts—1284
DTBench51%—
Epoch Capabilities Index119.27—
ForecastBench54.2—

Math Gemma 3n E4b IT leads

Claude 2.1: 10.2 (#315), Gemma 3n E4b IT: 35.1 (#188)

BenchmarkClaude 2.1Gemma 3n E4b IT
OTIS Mock AIME 2024-20251.9%—
LMArena Math—1251

Knowledge Gemma 3n E4b IT leads

Claude 2.1: 15.4 (#292), Gemma 3n E4b IT: 34.2 (#198)

BenchmarkClaude 2.1Gemma 3n E4b IT
GPQA Diamond33%—
LMArena Expert—1246
MMLU73.5%—

Multilingual Not comparable

Claude 2.1: —, Gemma 3n E4b IT: 43.4 (#183)

BenchmarkClaude 2.1Gemma 3n E4b IT
LMArena Non-English—1285
LMArena Chinese—1309
LMArena French—1330
LMArena German—1311
LMArena Japanese—1272
LMArena Korean—1259
LMArena Russian—1288
LMArena Spanish—1305

Instruction Following Not comparable

Claude 2.1: —, Gemma 3n E4b IT: 66.1 (#210)

BenchmarkClaude 2.1Gemma 3n E4b IT
LMArena Instruction Following—1255

Long Context Not comparable

Claude 2.1: —, Gemma 3n E4b IT: 38.7 (#191)

BenchmarkClaude 2.1Gemma 3n E4b IT
LMArena Longer Query—1276

Writing & Preference Not comparable

Claude 2.1: —, Gemma 3n E4b IT: 50.1 (#186)

BenchmarkClaude 2.1Gemma 3n E4b IT
LMArena Text—1306
LMArena Creative Writing—1287
LMArena Multi-Turn—1276

Frequently asked questions

Is Claude 2.1 better than Gemma 3n E4b IT?

Gemma 3n E4b IT is the stronger model overall, scoring 37.3 to 25.2 on the Noometry Index.

Is Claude 2.1 or Gemma 3n E4b IT better for coding?

Gemma 3n E4b IT scores higher on coding benchmarks: 37.0 versus 26.2 in the Noometry coding category.

How many benchmarks do Claude 2.1 and Gemma 3n E4b IT share?

0 benchmarks have published results for both models. Claude 2.1 has 7 scored results on Noometry and Gemma 3n E4b IT has 18.

Related comparisons

Go deeper