Noometry

Model comparison

Gemini 1.5 Flash (May 2024) vs GLM-4.7

GLM-4.7 is the stronger model overall, scoring 42.0 to 33.2 on the Noometry Index.

Last verified . 21 shared benchmarks.

Gemini 1.5 Flash (May 2024) Google

33.2

Rank #246 Confirmed

GLM-4.7 Z.ai (Zhipu)

42.0

Rank #124 Confirmed

Summary

  • They share 21 benchmarks with published results for both. Gemini 1.5 Flash (May 2024) scores higher in 1 category and GLM-4.7 in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where GLM-4.7 leads 47.0 to 26.2.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 16.3% for Gemini 1.5 Flash (May 2024) and 83.3% for GLM-4.7.
  • GLM-4.7 has downloadable open weights; the other is API-only.

Side by side

Gemini 1.5 Flash (May 2024)GLM-4.7
ProviderGoogleZ.ai (Zhipu)
Noometry Index33.242.0
Released2024-05-142025-12-22
WeightsProprietaryOpen
Context window—205K
Max output—131K
Input $ / M tokens—$0.60
Output $ / M tokens—$2.20
Results tracked4236

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GLM-4.7 leads

Gemini 1.5 Flash (May 2024): 34.4 (#236), GLM-4.7: 44.0 (#79)

BenchmarkGemini 1.5 Flash (May 2024)GLM-4.7
LMArena Coding12611454
LMArena WebDev—1435
SciCode—45.1%
WeirdML24.9%—
BigCodeBench Instruct43.5%—
BigCodeBench Complete55.1%—
ALE-Bench—399.48
HumanEval+75.6%—
MBPP+67.5%—

Agentic & Tool Use Too close to call

Gemini 1.5 Flash (May 2024): 26.6 (#102), GLM-4.7: 26.5 (#103)

BenchmarkGemini 1.5 Flash (May 2024)GLM-4.7
Terminal-Bench—33.4%
BALROG14.6%—
Vending-Bench 2—2,377

Reasoning GLM-4.7 leads

Gemini 1.5 Flash (May 2024): 21.7 (#215), GLM-4.7: 24.3 (#164)

BenchmarkGemini 1.5 Flash (May 2024)GLM-4.7
LMArena Hard Prompts12571443
Epoch Capabilities Index129.36143.51
SimpleBench—47.7%
CritPt—1.7%
Chess Puzzles—6%
DTBench53.8%—
ForecastBench53.9—
PIQA87.5%—

Math GLM-4.7 leads

Gemini 1.5 Flash (May 2024): 22.1 (#281), GLM-4.7: 38.6 (#135)

BenchmarkGemini 1.5 Flash (May 2024)GLM-4.7
OTIS Mock AIME 2024-202516.3%83.3%
LMArena Math12691423
FrontierMath (Feb 2025 set)0%2.4%
ProofBench—6%
Omni-MATH30.4%—
MATH Level 561.9%—
FrontierMath Tier 4 (v1)—0%
GSM8K82.4%—

Knowledge GLM-4.7 leads

Gemini 1.5 Flash (May 2024): 26.2 (#260), GLM-4.7: 47.0 (#80)

BenchmarkGemini 1.5 Flash (May 2024)GLM-4.7
GPQA Diamond47.3%83.3%
LMArena Expert12331424
SimpleQA Verified—32.2%
MMLU-Pro67.8%—
Vectara Hallucination Rate—11.7%
GPQA (HELM)43.7%—
BoolQ85.8%—
MMLU77.9%—

Multimodal Not comparable

Gemini 1.5 Flash (May 2024): 36.0 (#81), GLM-4.7: —

BenchmarkGemini 1.5 Flash (May 2024)GLM-4.7
LMArena Vision1141—
Video-MME70.3%—
GeoBench76%—

Multilingual GLM-4.7 leads

Gemini 1.5 Flash (May 2024): 42.9 (#189), GLM-4.7: 52.8 (#79)

BenchmarkGemini 1.5 Flash (May 2024)GLM-4.7
LMArena Non-English12781417
LMArena Chinese12951495
LMArena French12581432
LMArena German12621424
LMArena Japanese12521439
LMArena Korean12211399
LMArena Russian12881423
LMArena Spanish12431434

Instruction Following GLM-4.7 leads

Gemini 1.5 Flash (May 2024): 66.8 (#205), GLM-4.7: 74.4 (#95)

BenchmarkGemini 1.5 Flash (May 2024)GLM-4.7
LMArena Instruction Following12581411
IFEval83.1%—

Long Context GLM-4.7 leads

Gemini 1.5 Flash (May 2024): 39.0 (#187), GLM-4.7: 42.8 (#116)

BenchmarkGemini 1.5 Flash (May 2024)GLM-4.7
LMArena Longer Query12841432
CL-bench—15.9%
CL-bench Life—10.9%

Writing & Preference GLM-4.7 leads

Gemini 1.5 Flash (May 2024): 48.7 (#196), GLM-4.7: 60.9 (#93)

BenchmarkGemini 1.5 Flash (May 2024)GLM-4.7
LMArena Text12871435
LMArena Creative Writing12851401
LMArena Multi-Turn12531446
EQ-Bench Creative Writing—1413
WildBench79.2%—

Frequently asked questions

Is Gemini 1.5 Flash (May 2024) better than GLM-4.7?

GLM-4.7 is the stronger model overall, scoring 42.0 to 33.2 on the Noometry Index.

Is Gemini 1.5 Flash (May 2024) or GLM-4.7 better for coding?

GLM-4.7 scores higher on coding benchmarks: 44.0 versus 34.4 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash (May 2024) and GLM-4.7 share?

21 benchmarks have published results for both models. Gemini 1.5 Flash (May 2024) has 42 scored results on Noometry and GLM-4.7 has 36.

Related comparisons

Go deeper