Noometry

Model comparison

Gemma 4 26B A4B IT vs Llama 2-7B

Gemma 4 26B A4B IT is the stronger model overall, scoring 43.1 to 28.6 on the Noometry Index.

Last verified . 16 shared benchmarks.

Gemma 4 26B A4B IT Google

43.1

Rank #88 Confirmed

Llama 2-7B Meta

28.6

Rank #315 Confirmed

Summary

  • They share 16 benchmarks with published results for both. Gemma 4 26B A4B IT scores higher in 8 categories and Llama 2-7B in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Gemma 4 26B A4B IT leads 58.8 to 28.1.
  • The biggest single-benchmark swing is Chess Puzzles: 6% for Gemma 4 26B A4B IT and 0% for Llama 2-7B.

Side by side

Gemma 4 26B A4B ITLlama 2-7B
ProviderGoogleMeta
Noometry Index43.128.6
Released2026-04-022023-07-18
WeightsOpenOpen
Context window262K—
Max output33K—
Input $ / M tokens$0.0765—
Output $ / M tokens$0.26—
Results tracked2829

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 38.8 (#156), Llama 2-7B: 28.7 (#302)

BenchmarkGemma 4 26B A4B ITLlama 2-7B
LMArena Coding14461002
LMArena WebDev1359—
SciCode40%—
WeirdML35.2%—
ALE-Bench927.17—

Reasoning Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 21.1 (#205), Llama 2-7B: 15.1 (#309)

BenchmarkGemma 4 26B A4B ITLlama 2-7B
Chess Puzzles6%0%
LMArena Hard Prompts14381009
Epoch Capabilities Index141.8598.54
CritPt0%—
DTBench74.9%—
LMCA29.7%—
BIG-Bench Hard—39.2%
HellaSwag—77.2%
LAMBADA—73.3%
PIQA—78.8%
WinoGrande—69.2%

Math Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 45.7 (#61), Llama 2-7B: 28.2 (#231)

BenchmarkGemma 4 26B A4B ITLlama 2-7B
LMArena Math14691042
OTIS Mock AIME 2024-202582.2%—
GSM8K—16.7%

Knowledge Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 45.6 (#79), Llama 2-7B: 27.7 (#243)

BenchmarkGemma 4 26B A4B ITLlama 2-7B
LMArena Expert14471036
GPQA Diamond73.2%—
Vectara Hallucination Rate5.2%—
ARC (AI2) Challenge—45.9%
BoolQ—77.9%
MMLU—45.8%
OpenBookQA—58.6%
TriviaQA—73.7%

Multimodal Not comparable

Gemma 4 26B A4B IT: 39.9 (#43), Llama 2-7B: —

BenchmarkGemma 4 26B A4B ITLlama 2-7B
LMArena Vision1260—
ScienceQA—43.1%

Multilingual Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 53.3 (#68), Llama 2-7B: 23.9 (#288)

BenchmarkGemma 4 26B A4B ITLlama 2-7B
LMArena Non-English1422973
LMArena Chinese1495973
LMArena French1459969
LMArena Russian1434995
LMArena Spanish14151007
LMArena German—978

Instruction Following Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 75.0 (#78), Llama 2-7B: 51.0 (#293)

BenchmarkGemma 4 26B A4B ITLlama 2-7B
LMArena Instruction Following14201006

Long Context Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 43.6 (#87), Llama 2-7B: 30.4 (#282)

BenchmarkGemma 4 26B A4B ITLlama 2-7B
LMArena Longer Query1427999

Writing & Preference Gemma 4 26B A4B IT leads

Gemma 4 26B A4B IT: 58.8 (#110), Llama 2-7B: 28.1 (#293)

BenchmarkGemma 4 26B A4B ITLlama 2-7B
LMArena Text14341053
LMArena Creative Writing14011033
LMArena Multi-Turn14421029
EQ-Bench Creative Writing1305—

Frequently asked questions

Is Gemma 4 26B A4B IT better than Llama 2-7B?

Gemma 4 26B A4B IT is the stronger model overall, scoring 43.1 to 28.6 on the Noometry Index.

Is Gemma 4 26B A4B IT or Llama 2-7B better for coding?

Gemma 4 26B A4B IT scores higher on coding benchmarks: 38.8 versus 28.7 in the Noometry coding category.

How many benchmarks do Gemma 4 26B A4B IT and Llama 2-7B share?

16 benchmarks have published results for both models. Gemma 4 26B A4B IT has 28 scored results on Noometry and Llama 2-7B has 29.

Related comparisons

Go deeper