Noometry

Model comparison

Gemma 3 12B vs GLM-4.5-Air

GLM-4.5-Air is the stronger model overall, scoring 38.9 to 32.1 on the Noometry Index. Gemma 3 12B costs 5.7× less per token, which makes it the better buy when GLM-4.5-Air's lead doesn't matter for your workload.

Last verified . 13 shared benchmarks.

Gemma 3 12B Google

32.1

Rank #262 Confirmed

GLM-4.5-Air Z.ai (Zhipu)

38.9

Rank #177 Confirmed

Summary

  • They share 13 benchmarks with published results for both. Gemma 3 12B scores higher in 0 categories and GLM-4.5-Air in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where GLM-4.5-Air leads 36.2 to 22.3.
  • Gemma 3 12B is cheaper at $0.05 / $0.15 per million input/output tokens, against $0.20 / $1.10 for GLM-4.5-Air.

Side by side

Gemma 3 12BGLM-4.5-Air
ProviderGoogleZ.ai (Zhipu)
Noometry Index32.138.9
Released2025-03-122025-07-20
WeightsOpenOpen
Context window131K131K
Max output8K98K
Input $ / M tokens$0.05$0.20
Output $ / M tokens$0.15$1.10
Results tracked2427

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GLM-4.5-Air leads

Gemma 3 12B: 31.7 (#280), GLM-4.5-Air: 33.3 (#259)

BenchmarkGemma 3 12BGLM-4.5-Air
LMArena Coding12811397
SciCode17.4%—
GSO—2.9%

Agentic & Tool Use Not comparable

Gemma 3 12B: 25.5 (#108), GLM-4.5-Air: —

BenchmarkGemma 3 12BGLM-4.5-Air
Berkeley Function Calling Leaderboard30.4%—

Reasoning GLM-4.5-Air leads

Gemma 3 12B: 15.7 (#313), GLM-4.5-Air: 24.1 (#166)

BenchmarkGemma 3 12BGLM-4.5-Air
LMArena Hard Prompts13091379
Kagi LLM Benchmark—43%
CritPt0%—
Chess Puzzles0%—
DTBench48.8%—
LMCA4.5%—
Epoch Capabilities Index123.5—
ForecastBench—59.2

Math GLM-4.5-Air leads

Gemma 3 12B: 22.3 (#279), GLM-4.5-Air: 36.2 (#170)

BenchmarkGemma 3 12BGLM-4.5-Air
LMArena Math13071396
OTIS Mock AIME 2024-202516.7%—
Omni-MATH—39.1%

Knowledge GLM-4.5-Air leads

Gemma 3 12B: 26.5 (#257), GLM-4.5-Air: 35.0 (#191)

BenchmarkGemma 3 12BGLM-4.5-Air
Vectara Hallucination Rate4.4%9.3%
LMArena Expert12481370
GPQA Diamond39.5%—
Humanity's Last Exam—8.1%
MMLU-Pro—76.2%
GPQA (HELM)—59.4%

Multimodal Not comparable

Gemma 3 12B: —, GLM-4.5-Air: —

BenchmarkGemma 3 12BGLM-4.5-Air
MindCube46.7%—

Multilingual GLM-4.5-Air leads

Gemma 3 12B: 45.7 (#165), GLM-4.5-Air: 49.1 (#135)

BenchmarkGemma 3 12BGLM-4.5-Air
LMArena Non-English13181366
LMArena German13701377
LMArena Russian13351373
LMArena Chinese—1426
LMArena French—1399
LMArena Japanese—1348
LMArena Korean—1308
LMArena Spanish—1386

Instruction Following GLM-4.5-Air leads

Gemma 3 12B: 68.6 (#186), GLM-4.5-Air: 69.6 (#171)

BenchmarkGemma 3 12BGLM-4.5-Air
LMArena Instruction Following12991354
IFEval—81.2%

Long Context GLM-4.5-Air leads

Gemma 3 12B: 40.0 (#162), GLM-4.5-Air: 41.6 (#135)

BenchmarkGemma 3 12BGLM-4.5-Air
LMArena Longer Query13171366

Writing & Preference GLM-4.5-Air leads

Gemma 3 12B: 47.5 (#209), GLM-4.5-Air: 55.9 (#139)

BenchmarkGemma 3 12BGLM-4.5-Air
LMArena Text13341384
LMArena Creative Writing13311343
LMArena Multi-Turn13341371
EQ-Bench Creative Writing1126—
WildBench—78.9%

Frequently asked questions

Is Gemma 3 12B better than GLM-4.5-Air?

GLM-4.5-Air is the stronger model overall, scoring 38.9 to 32.1 on the Noometry Index. Gemma 3 12B costs 5.7× less per token, which makes it the better buy when GLM-4.5-Air's lead doesn't matter for your workload.

Which is cheaper, Gemma 3 12B or GLM-4.5-Air?

Gemma 3 12B is cheaper. It lists at $0.05 per million input tokens and $0.15 per million output tokens; GLM-4.5-Air lists at $0.20 and $1.10.

Is Gemma 3 12B or GLM-4.5-Air better for coding?

GLM-4.5-Air scores higher on coding benchmarks: 33.3 versus 31.7 in the Noometry coding category.

Which has the bigger context window?

Both accept 131K tokens.

How many benchmarks do Gemma 3 12B and GLM-4.5-Air share?

13 benchmarks have published results for both models. Gemma 3 12B has 24 scored results on Noometry and GLM-4.5-Air has 27.

Related comparisons

Go deeper