Noometry

Model comparison

Gemma 1.1 7b IT vs Palm 2

Gemma 1.1 7b IT is the stronger model overall, scoring 30.7 to 29.4 on the Noometry Index.

Last verified . 10 shared benchmarks.

Gemma 1.1 7b IT Google

30.7

Rank #275 Confirmed

Palm 2 Google

29.4

Rank #296 Confirmed

Summary

  • They share 10 benchmarks with published results for both. Gemma 1.1 7b IT scores higher in 7 categories and Palm 2 in 0 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in multilingual, where Gemma 1.1 7b IT leads 28.3 to 21.1.
  • Gemma 1.1 7b IT has downloadable open weights; the other is API-only.

Side by side

Gemma 1.1 7b ITPalm 2
ProviderGoogleGoogle
Noometry Index30.729.4
Released——
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked1910

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemma 1.1 7b IT leads

Gemma 1.1 7b IT: 31.0 (#279), Palm 2: 28.5 (#306)

BenchmarkGemma 1.1 7b ITPalm 2
LMArena Coding1084995
HumanEval+35.4%—
MBPP+45%—

Reasoning Gemma 1.1 7b IT leads

Gemma 1.1 7b IT: 19.6 (#235), Palm 2: 18.2 (#270)

BenchmarkGemma 1.1 7b ITPalm 2
LMArena Hard Prompts10711005

Math Gemma 1.1 7b IT leads

Gemma 1.1 7b IT: 29.5 (#217), Palm 2: 28.4 (#229)

BenchmarkGemma 1.1 7b ITPalm 2
LMArena Math11071049

Knowledge Not comparable

Gemma 1.1 7b IT: 27.8 (#242), Palm 2: —

BenchmarkGemma 1.1 7b ITPalm 2
LMArena Expert1039—

Multilingual Gemma 1.1 7b IT leads

Gemma 1.1 7b IT: 28.3 (#268), Palm 2: 21.1 (#290)

BenchmarkGemma 1.1 7b ITPalm 2
LMArena Non-English1052916
LMArena Chinese1060887
LMArena French1064—
LMArena German1054—
LMArena Japanese971—
LMArena Korean988—
LMArena Russian1046—
LMArena Spanish1049—

Instruction Following Gemma 1.1 7b IT leads

Gemma 1.1 7b IT: 54.3 (#278), Palm 2: 51.3 (#291)

BenchmarkGemma 1.1 7b ITPalm 2
LMArena Instruction Following10571010

Long Context Gemma 1.1 7b IT leads

Gemma 1.1 7b IT: 32.0 (#267), Palm 2: 30.7 (#280)

BenchmarkGemma 1.1 7b ITPalm 2
LMArena Longer Query10561010

Writing & Preference Gemma 1.1 7b IT leads

Gemma 1.1 7b IT: 30.6 (#283), Palm 2: 25.7 (#299)

BenchmarkGemma 1.1 7b ITPalm 2
LMArena Text10941027
LMArena Creative Writing1059991
LMArena Multi-Turn1040994

Frequently asked questions

Is Gemma 1.1 7b IT better than Palm 2?

Gemma 1.1 7b IT is the stronger model overall, scoring 30.7 to 29.4 on the Noometry Index.

Is Gemma 1.1 7b IT or Palm 2 better for coding?

Gemma 1.1 7b IT scores higher on coding benchmarks: 31.0 versus 28.5 in the Noometry coding category.

How many benchmarks do Gemma 1.1 7b IT and Palm 2 share?

10 benchmarks have published results for both models. Gemma 1.1 7b IT has 19 scored results on Noometry and Palm 2 has 10.

Related comparisons

Go deeper