Noometry

Model comparison

Gemma 2 9B vs GLM-5.1

GLM-5.1 is the stronger model overall, scoring 47.6 to 25.7 on the Noometry Index.

Last verified . 21 shared benchmarks.

Gemma 2 9B Google

25.7

Rank #334 Confirmed

GLM-5.1 Z.ai (Zhipu)

47.6

Rank #51 Confirmed

Summary

  • They share 21 benchmarks with published results for both. Gemma 2 9B scores higher in 0 categories and GLM-5.1 in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where GLM-5.1 leads 55.0 to 9.7.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 0.6% for Gemma 2 9B and 93.3% for GLM-5.1.

Side by side

Gemma 2 9BGLM-5.1
ProviderGoogleZ.ai (Zhipu)
Noometry Index25.747.6
Released2024-06-242026-04-07
WeightsOpenOpen
Context window—200K
Max output—131K
Input $ / M tokens—$1.40
Output $ / M tokens—$4.40
Results tracked3541

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GLM-5.1 leads

Gemma 2 9B: 29.0 (#299), GLM-5.1: 48.5 (#52)

BenchmarkGemma 2 9BGLM-5.1
LMArena Coding11741486
SWE-bench Verified—74.2%
LMArena WebDev—1509
SciCode—43.8%
WeirdML—57.1%
BigCodeBench Instruct34.7%—
LiveBench Coding22.5%—
BigCodeBench Complete40.6%—
ALE-Bench—887.1

Agentic & Tool Use Not comparable

Gemma 2 9B: —, GLM-5.1: 24.8 (#106)

BenchmarkGemma 2 9BGLM-5.1
APEX-Agents—40.9%
ExploitBench—18.1%
GBAEval—0%
Vending-Bench 2—5,634

Reasoning GLM-5.1 leads

Gemma 2 9B: 15.3 (#306), GLM-5.1: 38.2 (#57)

BenchmarkGemma 2 9BGLM-5.1
LMArena Hard Prompts11721472
Epoch Capabilities Index119.78149.86
SimpleBench—55.1%
NYT Connections (extended)—77.7%
CritPt—4.6%
Chess Puzzles—19%
Thematic Generalization—69.8%
LiveBench Reasoning15.2%—
LiveBench Data Analysis36.4%—
LiveBench28.7%—
PIQA83.7%—

Math GLM-5.1 leads

Gemma 2 9B: 9.3 (#312), GLM-5.1: 49.0 (#48)

BenchmarkGemma 2 9BGLM-5.1
OTIS Mock AIME 2024-20250.6%93.3%
LMArena Math11831472
FrontierMath (Tiers 1-3)—36.8%
MathArena Final-Answer Competitions—67.1%
ProofBench—22.2%
LiveBench Math19.8%—
MATH Level 521%—
FrontierMath (Feb 2025 set)—33.4%
FrontierMath Tier 4 (v1)—12.5%
GSM8K84.9%—

Knowledge GLM-5.1 leads

Gemma 2 9B: 9.7 (#300), GLM-5.1: 55.0 (#45)

BenchmarkGemma 2 9BGLM-5.1
GPQA Diamond27.5%89.9%
LMArena Expert11471477
SimpleQA Verified—34%
BoolQ85.7%—
MMLU72.1%—

Multilingual GLM-5.1 leads

Gemma 2 9B: 36.8 (#233), GLM-5.1: 55.2 (#34)

BenchmarkGemma 2 9BGLM-5.1
LMArena Non-English11881448
LMArena Chinese11851516
LMArena French11891478
LMArena German11861466
LMArena Japanese11441436
LMArena Korean11371423
LMArena Russian12001453
LMArena Spanish12001467

Instruction Following GLM-5.1 leads

Gemma 2 9B: 57.8 (#264), GLM-5.1: 76.5 (#41)

BenchmarkGemma 2 9BGLM-5.1
LMArena Instruction Following11781451
LiveBench Instruction Following52.6%—

Long Context GLM-5.1 leads

Gemma 2 9B: 36.3 (#228), GLM-5.1: 44.9 (#51)

BenchmarkGemma 2 9BGLM-5.1
LMArena Longer Query11971467

Writing & Preference GLM-5.1 leads

Gemma 2 9B: 32.3 (#276), GLM-5.1: 67.2 (#29)

BenchmarkGemma 2 9BGLM-5.1
LMArena Text12071462
LMArena Creative Writing12061454
EQ-Bench Creative Writing8411592
LMArena Multi-Turn11931473
LiveBench Language25.5%—

Frequently asked questions

Is Gemma 2 9B better than GLM-5.1?

GLM-5.1 is the stronger model overall, scoring 47.6 to 25.7 on the Noometry Index.

Is Gemma 2 9B or GLM-5.1 better for coding?

GLM-5.1 scores higher on coding benchmarks: 48.5 versus 29.0 in the Noometry coding category.

How many benchmarks do Gemma 2 9B and GLM-5.1 share?

21 benchmarks have published results for both models. Gemma 2 9B has 35 scored results on Noometry and GLM-5.1 has 41.

Related comparisons

Go deeper