Noometry

Model comparison

Gemma 1.1 7b IT vs Phi-3.5-mini

Gemma 1.1 7b IT has enough public results to be ranked (#277); Phi-3.5-mini does not yet, so treat this comparison as directional.

Last verified . 0 shared benchmarks.

Gemma 1.1 7b IT Google

31.3

Rank #277 Confirmed

Phi-3.5-mini Microsoft

36.9

Unranked Sparse

Summary

  • The widest gap is in coding, where Phi-3.5-mini leads 34.6 to 31.5.

Side by side

Gemma 1.1 7b ITPhi-3.5-mini
ProviderGoogleMicrosoft
Noometry Index31.336.9
Released—2024-08-16
WeightsOpenOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked195

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Phi-3.5-mini leads

Gemma 1.1 7b IT: 31.5 (#284), Phi-3.5-mini: 34.6 (#232)

BenchmarkGemma 1.1 7b ITPhi-3.5-mini
BigCodeBench Instruct—32.8%
LMArena Coding1084—
BigCodeBench Complete—38.5%
HumanEval+35.4%—
MBPP+45%—

Reasoning Not comparable

Gemma 1.1 7b IT: 20.5 (#238), Phi-3.5-mini: —

BenchmarkGemma 1.1 7b ITPhi-3.5-mini
LMArena Hard Prompts1071—
PIQA—81%

Math Not comparable

Gemma 1.1 7b IT: 32.0 (#220), Phi-3.5-mini: —

BenchmarkGemma 1.1 7b ITPhi-3.5-mini
LMArena Math1107—
GSM8K—86.2%

Knowledge Not comparable

Gemma 1.1 7b IT: 28.3 (#247), Phi-3.5-mini: —

BenchmarkGemma 1.1 7b ITPhi-3.5-mini
LMArena Expert1039—
BoolQ—78%

Multilingual Not comparable

Gemma 1.1 7b IT: 28.1 (#273), Phi-3.5-mini: —

BenchmarkGemma 1.1 7b ITPhi-3.5-mini
LMArena Non-English1052—
LMArena Chinese1061—
LMArena French1065—
LMArena German1054—
LMArena Japanese971—
LMArena Korean988—
LMArena Russian1046—
LMArena Spanish1049—

Instruction Following Not comparable

Gemma 1.1 7b IT: 54.0 (#283), Phi-3.5-mini: —

BenchmarkGemma 1.1 7b ITPhi-3.5-mini
LMArena Instruction Following1057—

Long Context Not comparable

Gemma 1.1 7b IT: 32.1 (#272), Phi-3.5-mini: —

BenchmarkGemma 1.1 7b ITPhi-3.5-mini
LMArena Longer Query1056—

Writing & Preference Not comparable

Gemma 1.1 7b IT: 30.4 (#288), Phi-3.5-mini: —

BenchmarkGemma 1.1 7b ITPhi-3.5-mini
LMArena Text1094—
LMArena Creative Writing1060—
LMArena Multi-Turn1040—

Frequently asked questions

Is Gemma 1.1 7b IT better than Phi-3.5-mini?

Gemma 1.1 7b IT has enough public results to be ranked (#277); Phi-3.5-mini does not yet, so treat this comparison as directional.

Is Gemma 1.1 7b IT or Phi-3.5-mini better for coding?

Phi-3.5-mini scores higher on coding benchmarks: 34.6 versus 31.5 in the Noometry coding category.

How many benchmarks do Gemma 1.1 7b IT and Phi-3.5-mini share?

0 benchmarks have published results for both models. Gemma 1.1 7b IT has 19 scored results on Noometry and Phi-3.5-mini has 5.

Related comparisons

Go deeper