Noometry

Model comparison

GLM-4.5-Air vs Grok-2 (Dec 2024)

GLM-4.5-Air is the stronger model overall, scoring 38.7 to 33.4 on the Noometry Index.

Last verified . 17 shared benchmarks.

GLM-4.5-Air Z.ai (Zhipu)

38.7

Rank #169 Confirmed

Grok-2 (Dec 2024) xAI

33.4

Rank #233 Confirmed

Summary

  • They share 17 benchmarks with published results for both. GLM-4.5-Air scores higher in 8 categories and Grok-2 (Dec 2024) in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where GLM-4.5-Air leads 33.8 to 19.6.
  • GLM-4.5-Air has downloadable open weights; the other is API-only.

Side by side

GLM-4.5-AirGrok-2 (Dec 2024)
ProviderZ.ai (Zhipu)xAI
Noometry Index38.733.4
Released2025-07-202024-08-13
WeightsOpenProprietary
Context window131K—
Max output98K—
Input $ / M tokens$0.20—
Output $ / M tokens$1.10—
Results tracked2734

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GLM-4.5-Air leads

GLM-4.5-Air: 34.5 (#225), Grok-2 (Dec 2024): 32.9 (#254)

BenchmarkGLM-4.5-AirGrok-2 (Dec 2024)
LMArena Coding13971288
GSO2.9%—
WeirdML—22.2%
LiveBench Coding—46.4%

Reasoning GLM-4.5-Air leads

GLM-4.5-Air: 23.3 (#166), Grok-2 (Dec 2024): 16.4 (#294)

BenchmarkGLM-4.5-AirGrok-2 (Dec 2024)
LMArena Hard Prompts13791272
SimpleBench—22.7%
Kagi LLM Benchmark43%—
LiveBench Reasoning—54.8%
DTBench—65.2%
LiveBench Data Analysis—54.5%
Epoch Capabilities Index—130.48
ForecastBench59.2—
LiveBench—54.3%

Math GLM-4.5-Air leads

GLM-4.5-Air: 33.8 (#166), Grok-2 (Dec 2024): 19.6 (#278)

BenchmarkGLM-4.5-AirGrok-2 (Dec 2024)
LMArena Math13961283
OTIS Mock AIME 2024-2025—11.5%
Omni-MATH39.1%—
LiveBench Math—54.9%
MATH Level 5—63.5%
FrontierMath (Feb 2025 set)—0.7%

Knowledge GLM-4.5-Air leads

GLM-4.5-Air: 34.6 (#184), Grok-2 (Dec 2024): 29.8 (#225)

BenchmarkGLM-4.5-AirGrok-2 (Dec 2024)
LMArena Expert13701254
GPQA Diamond—53.8%
Humanity's Last Exam8.1%—
MMLU-Pro76.2%—
Confabulations—20.1%
Vectara Hallucination Rate9.3%—
GPQA (HELM)59.4%—

Multilingual GLM-4.5-Air leads

GLM-4.5-Air: 49.4 (#130), Grok-2 (Dec 2024): 43.3 (#183)

BenchmarkGLM-4.5-AirGrok-2 (Dec 2024)
LMArena Non-English13661282
LMArena Chinese14271289
LMArena French13991317
LMArena German13761287
LMArena Japanese13481244
LMArena Korean13061237
LMArena Russian13731286
LMArena Spanish13851281

Instruction Following GLM-4.5-Air leads

GLM-4.5-Air: 69.8 (#165), Grok-2 (Dec 2024): 67.1 (#197)

BenchmarkGLM-4.5-AirGrok-2 (Dec 2024)
LMArena Instruction Following13531270
LiveBench Instruction Following—69.6%
IFEval81.2%—

Long Context GLM-4.5-Air leads

GLM-4.5-Air: 41.6 (#130), Grok-2 (Dec 2024): 38.7 (#186)

BenchmarkGLM-4.5-AirGrok-2 (Dec 2024)
LMArena Longer Query13671276

Writing & Preference GLM-4.5-Air leads

GLM-4.5-Air: 56.1 (#134), Grok-2 (Dec 2024): 48.8 (#192)

BenchmarkGLM-4.5-AirGrok-2 (Dec 2024)
LMArena Text13841305
LMArena Creative Writing13431284
LMArena Multi-Turn13711290
Short-Story Creative Writing—63.6%
WildBench78.9%—
LiveBench Language—45.6%

Frequently asked questions

Is GLM-4.5-Air better than Grok-2 (Dec 2024)?

GLM-4.5-Air is the stronger model overall, scoring 38.7 to 33.4 on the Noometry Index.

Is GLM-4.5-Air or Grok-2 (Dec 2024) better for coding?

GLM-4.5-Air scores higher on coding benchmarks: 34.5 versus 32.9 in the Noometry coding category.

How many benchmarks do GLM-4.5-Air and Grok-2 (Dec 2024) share?

17 benchmarks have published results for both models. GLM-4.5-Air has 27 scored results on Noometry and Grok-2 (Dec 2024) has 34.

Related comparisons

Go deeper