Noometry

Model comparison

Gemini 1.5 Pro (May 2024) vs GLM-4.6V

GLM-4.6V is the stronger model overall, scoring 40.9 to 31.9 on the Noometry Index.

Last verified . 12 shared benchmarks.

Gemini 1.5 Pro (May 2024) Google

31.9

Rank #253 Confirmed

GLM-4.6V Z.ai (Zhipu)

40.9

Rank #133 Confirmed

Summary

  • They share 12 benchmarks with published results for both. Gemini 1.5 Pro (May 2024) scores higher in 1 category and GLM-4.6V in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where GLM-4.6V leads 26.5 to 12.3.
  • GLM-4.6V has downloadable open weights; the other is API-only.

Side by side

Gemini 1.5 Pro (May 2024)GLM-4.6V
ProviderGoogleZ.ai (Zhipu)
Noometry Index31.940.9
Released2024-02-152025-12-08
WeightsProprietaryOpen
Context window—128K
Max output—33K
Input $ / M tokens—$0.30
Output $ / M tokens—$0.90
Results tracked4512

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GLM-4.6V leads

Gemini 1.5 Pro (May 2024): 33.8 (#239), GLM-4.6V: 40.2 (#128)

BenchmarkGemini 1.5 Pro (May 2024)GLM-4.6V
LMArena Coding12941390
WeirdML22.2%—
BigCodeBench Instruct43.8%—
BigCodeBench Complete57.5%—
CadEval34%—
HumanEval+79.3%—
MBPP+74.6%—

Agentic & Tool Use Not comparable

Gemini 1.5 Pro (May 2024): 17.8 (#137), GLM-4.6V: —

BenchmarkGemini 1.5 Pro (May 2024)GLM-4.6V
TheAgentCompany3.4%—
Cybench7.5%—
BALROG21%—

Reasoning GLM-4.6V leads

Gemini 1.5 Pro (May 2024): 12.3 (#332), GLM-4.6V: 26.5 (#117)

BenchmarkGemini 1.5 Pro (May 2024)GLM-4.6V
LMArena Hard Prompts12961368
ARC-AGI-20.8%—
SimpleBench27.1%—
DTBench59%—
BIG-Bench Hard89.2%—
Epoch Capabilities Index131.73—
ForecastBench58.4—

Math Not comparable

Gemini 1.5 Pro (May 2024): 24.4 (#260), GLM-4.6V: —

BenchmarkGemini 1.5 Pro (May 2024)GLM-4.6V
OTIS Mock AIME 2024-202523.1%—
Omni-MATH36.4%—
LMArena Math1315—
MATH Level 570.4%—

Knowledge GLM-4.6V leads

Gemini 1.5 Pro (May 2024): 29.3 (#228), GLM-4.6V: 37.4 (#148)

BenchmarkGemini 1.5 Pro (May 2024)GLM-4.6V
LMArena Expert12791372
GPQA Diamond57.2%—
Humanity's Last Exam4.6%—
MMLU-Pro73.7%—
Confabulations13.5%—
GPQA (HELM)53.4%—
MMLU86.9%—

Multimodal Gemini 1.5 Pro (May 2024) leads

Gemini 1.5 Pro (May 2024): 36.2 (#72), GLM-4.6V: 34.2 (#83)

BenchmarkGemini 1.5 Pro (May 2024)GLM-4.6V
LMArena Vision11611164
Video-MME75%—

Multilingual GLM-4.6V leads

Gemini 1.5 Pro (May 2024): 45.5 (#169), GLM-4.6V: 48.8 (#135)

BenchmarkGemini 1.5 Pro (May 2024)GLM-4.6V
LMArena Non-English13121359
LMArena Chinese13311426
LMArena Russian13201341
LMArena French1301—
LMArena German1286—
LMArena Japanese1292—
LMArena Korean1298—
LMArena Spanish1310—

Instruction Following GLM-4.6V leads

Gemini 1.5 Pro (May 2024): 68.8 (#180), GLM-4.6V: 71.6 (#145)

BenchmarkGemini 1.5 Pro (May 2024)GLM-4.6V
LMArena Instruction Following12971351
IFEval83.7%—

Long Context GLM-4.6V leads

Gemini 1.5 Pro (May 2024): 39.7 (#164), GLM-4.6V: 41.3 (#138)

BenchmarkGemini 1.5 Pro (May 2024)GLM-4.6V
LMArena Longer Query13081358

Writing & Preference GLM-4.6V leads

Gemini 1.5 Pro (May 2024): 52.5 (#167), GLM-4.6V: 56.8 (#132)

BenchmarkGemini 1.5 Pro (May 2024)GLM-4.6V
LMArena Text13191377
LMArena Creative Writing13331347
LMArena Multi-Turn12961359
WildBench81.3%—

Frequently asked questions

Is Gemini 1.5 Pro (May 2024) better than GLM-4.6V?

GLM-4.6V is the stronger model overall, scoring 40.9 to 31.9 on the Noometry Index.

Is Gemini 1.5 Pro (May 2024) or GLM-4.6V better for coding?

GLM-4.6V scores higher on coding benchmarks: 40.2 versus 33.8 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Pro (May 2024) and GLM-4.6V share?

12 benchmarks have published results for both models. Gemini 1.5 Pro (May 2024) has 45 scored results on Noometry and GLM-4.6V has 12.

Related comparisons

Go deeper