Noometry

Model comparison

Gemma 2 9B vs Qwen2.5 72B Instruct

Qwen2.5 72B Instruct is the stronger model overall, scoring 31.9 to 25.9 on the Noometry Index.

Last verified . 25 shared benchmarks.

Gemma 2 9B Google

25.9

Rank #341 Confirmed

Qwen2.5 72B Instruct Alibaba (Qwen)

31.9

Rank #267 Confirmed

Summary

  • They share 25 benchmarks with published results for both. Gemma 2 9B scores higher in 0 categories and Qwen2.5 72B Instruct in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Qwen2.5 72B Instruct leads 27.0 to 9.7.
  • The biggest single-benchmark swing is MATH Level 5: 21% for Gemma 2 9B and 63.2% for Qwen2.5 72B Instruct.

Side by side

Gemma 2 9BQwen2.5 72B Instruct
ProviderGoogleAlibaba (Qwen)
Noometry Index25.931.9
Released2024-06-242024-09
WeightsOpenOpen
Context window—131K
Max output—8K
Input $ / M tokens—$1.40
Output $ / M tokens—$5.60
Results tracked3543

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5 72B Instruct leads

Gemma 2 9B: 29.4 (#304), Qwen2.5 72B Instruct: 33.2 (#260)

BenchmarkGemma 2 9BQwen2.5 72B Instruct
BigCodeBench Instruct34.7%45.8%
LMArena Coding11731292
BigCodeBench Complete40.6%55.9%
WeirdML—16%
LiveBench Coding22.5%—

Agentic & Tool Use Not comparable

Gemma 2 9B: —, Qwen2.5 72B Instruct: 22.1 (#133)

BenchmarkGemma 2 9BQwen2.5 72B Instruct
TheAgentCompany—5.7%
BALROG—16.2%
METR Time Horizons—35.8%

Reasoning Qwen2.5 72B Instruct leads

Gemma 2 9B: 15.9 (#309), Qwen2.5 72B Instruct: 22.3 (#199)

BenchmarkGemma 2 9BQwen2.5 72B Instruct
LMArena Hard Prompts11711271
Epoch Capabilities Index119.83129
PIQA83.7%82.6%
LiveBench Reasoning15.2%—
DTBench—62.9%
LiveBench Data Analysis36.4%—
LMCA—13.4%
BIG-Bench Hard—79.8%
ForecastBench—57.5
HellaSwag—84.8%
LiveBench28.7%—
WinoGrande—82.3%

Math Qwen2.5 72B Instruct leads

Gemma 2 9B: 9.9 (#318), Qwen2.5 72B Instruct: 19.3 (#287)

BenchmarkGemma 2 9BQwen2.5 72B Instruct
OTIS Mock AIME 2024-20250.6%8.1%
LMArena Math11831283
MATH Level 521%63.2%
Omni-MATH—33%
LiveBench Math19.8%—
GSM8K84.9%—

Knowledge Qwen2.5 72B Instruct leads

Gemma 2 9B: 9.7 (#305), Qwen2.5 72B Instruct: 27.0 (#253)

BenchmarkGemma 2 9BQwen2.5 72B Instruct
GPQA Diamond27.5%49.1%
LMArena Expert11471245
MMLU72.1%85.3%
MMLU-Pro—63.1%
Confabulations—19.1%
GPQA (HELM)—42.6%
ARC (AI2) Challenge—94.5%
BoolQ85.7%—
TriviaQA—71.9%

Multilingual Qwen2.5 72B Instruct leads

Gemma 2 9B: 36.6 (#238), Qwen2.5 72B Instruct: 41.0 (#213)

BenchmarkGemma 2 9BQwen2.5 72B Instruct
LMArena Non-English11881252
LMArena Chinese11851272
LMArena French11901280
LMArena German11861234
LMArena Japanese11441180
LMArena Korean11371188
LMArena Russian12001264
LMArena Spanish12001256

Instruction Following Qwen2.5 72B Instruct leads

Gemma 2 9B: 57.6 (#269), Qwen2.5 72B Instruct: 65.5 (#221)

BenchmarkGemma 2 9BQwen2.5 72B Instruct
LMArena Instruction Following11781254
LiveBench Instruction Following52.6%—
IFEval—80.6%

Long Context Qwen2.5 72B Instruct leads

Gemma 2 9B: 36.3 (#233), Qwen2.5 72B Instruct: 38.9 (#188)

BenchmarkGemma 2 9BQwen2.5 72B Instruct
LMArena Longer Query11971282

Writing & Preference Qwen2.5 72B Instruct leads

Gemma 2 9B: 32.1 (#281), Qwen2.5 72B Instruct: 46.7 (#215)

BenchmarkGemma 2 9BQwen2.5 72B Instruct
LMArena Text12071269
LMArena Creative Writing12061221
LMArena Multi-Turn11931272
EQ-Bench Creative Writing841—
WildBench—80.2%
LiveBench Language25.5%—

Frequently asked questions

Is Gemma 2 9B better than Qwen2.5 72B Instruct?

Qwen2.5 72B Instruct is the stronger model overall, scoring 31.9 to 25.9 on the Noometry Index.

Is Gemma 2 9B or Qwen2.5 72B Instruct better for coding?

Qwen2.5 72B Instruct scores higher on coding benchmarks: 33.2 versus 29.4 in the Noometry coding category.

How many benchmarks do Gemma 2 9B and Qwen2.5 72B Instruct share?

25 benchmarks have published results for both models. Gemma 2 9B has 35 scored results on Noometry and Qwen2.5 72B Instruct has 43.

Related comparisons

Go deeper