Noometry

Model comparison

Gemma 3 4B vs GLM-4.5-Air

GLM-4.5-Air is the stronger model overall, scoring 38.7 to 27.9 on the Noometry Index. Gemma 3 4B costs 8.5× less per token, which makes it the better buy when GLM-4.5-Air's lead doesn't matter for your workload.

Last verified . 14 shared benchmarks.

Gemma 3 4B Google

27.9

Rank #320 Confirmed

GLM-4.5-Air Z.ai (Zhipu)

38.7

Rank #169 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Gemma 3 4B scores higher in 1 category and GLM-4.5-Air in 7 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where GLM-4.5-Air leads 34.6 to 11.8.
  • The biggest single-benchmark swing is Kagi LLM Benchmark: 25.2% for Gemma 3 4B and 43% for GLM-4.5-Air.
  • Gemma 3 4B is cheaper at $0.04 / $0.08 per million input/output tokens, against $0.20 / $1.10 for GLM-4.5-Air.

Side by side

Gemma 3 4BGLM-4.5-Air
ProviderGoogleZ.ai (Zhipu)
Noometry Index27.938.7
Released2025-03-122025-07-20
WeightsOpenOpen
Context window131K131K
Max output4K98K
Input $ / M tokens$0.04$0.20
Output $ / M tokens$0.08$1.10
Results tracked2227

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

Gemma 3 4B: 35.3 (#214), GLM-4.5-Air: 34.5 (#225)

BenchmarkGemma 3 4BGLM-4.5-Air
LMArena Coding12301397
GSO—2.9%

Agentic & Tool Use Not comparable

Gemma 3 4B: 20.7 (#134), GLM-4.5-Air: —

BenchmarkGemma 3 4BGLM-4.5-Air
Berkeley Function Calling Leaderboard19.6%—

Reasoning GLM-4.5-Air leads

Gemma 3 4B: 12.7 (#330), GLM-4.5-Air: 23.3 (#166)

BenchmarkGemma 3 4BGLM-4.5-Air
Kagi LLM Benchmark25.2%43%
LMArena Hard Prompts12531379
Chess Puzzles0%—
DTBench50.9%—
LMCA2.8%—
Epoch Capabilities Index116.02—
ForecastBench—59.2

Math GLM-4.5-Air leads

Gemma 3 4B: 15.8 (#286), GLM-4.5-Air: 33.8 (#166)

BenchmarkGemma 3 4BGLM-4.5-Air
LMArena Math12391396
OTIS Mock AIME 2024-20257.5%—
Omni-MATH—39.1%

Knowledge GLM-4.5-Air leads

Gemma 3 4B: 11.8 (#294), GLM-4.5-Air: 34.6 (#184)

BenchmarkGemma 3 4BGLM-4.5-Air
Vectara Hallucination Rate6.4%9.3%
LMArena Expert12231370
GPQA Diamond23.2%—
Humanity's Last Exam—8.1%
MMLU-Pro—76.2%
GPQA (HELM)—59.4%

Multilingual GLM-4.5-Air leads

Gemma 3 4B: 42.7 (#189), GLM-4.5-Air: 49.4 (#130)

BenchmarkGemma 3 4BGLM-4.5-Air
LMArena Non-English12731366
LMArena German12811376
LMArena Russian12941373
LMArena Chinese—1427
LMArena French—1399
LMArena Japanese—1348
LMArena Korean—1306
LMArena Spanish—1385

Instruction Following GLM-4.5-Air leads

Gemma 3 4B: 65.4 (#219), GLM-4.5-Air: 69.8 (#165)

BenchmarkGemma 3 4BGLM-4.5-Air
LMArena Instruction Following12391353
IFEval—81.2%

Long Context GLM-4.5-Air leads

Gemma 3 4B: 38.6 (#189), GLM-4.5-Air: 41.6 (#130)

BenchmarkGemma 3 4BGLM-4.5-Air
LMArena Longer Query12741367

Writing & Preference GLM-4.5-Air leads

Gemma 3 4B: 42.2 (#234), GLM-4.5-Air: 56.1 (#134)

BenchmarkGemma 3 4BGLM-4.5-Air
LMArena Text12911384
LMArena Creative Writing12701343
LMArena Multi-Turn12551371
EQ-Bench Creative Writing1068—
WildBench—78.9%

Frequently asked questions

Is Gemma 3 4B better than GLM-4.5-Air?

GLM-4.5-Air is the stronger model overall, scoring 38.7 to 27.9 on the Noometry Index. Gemma 3 4B costs 8.5× less per token, which makes it the better buy when GLM-4.5-Air's lead doesn't matter for your workload.

Which is cheaper, Gemma 3 4B or GLM-4.5-Air?

Gemma 3 4B is cheaper. It lists at $0.04 per million input tokens and $0.08 per million output tokens; GLM-4.5-Air lists at $0.20 and $1.10.

Is Gemma 3 4B or GLM-4.5-Air better for coding?

They score almost the same on coding (35.3 vs 34.5); test both on your own repository before choosing.

Which has the bigger context window?

Both accept 131K tokens.

How many benchmarks do Gemma 3 4B and GLM-4.5-Air share?

14 benchmarks have published results for both models. Gemma 3 4B has 22 scored results on Noometry and GLM-4.5-Air has 27.

Related comparisons

Go deeper