Noometry

Model comparison

Gemma 2 9B vs Qwen3.5 397B-A17B

Qwen3.5 397B-A17B is the stronger model overall, scoring 45.7 to 25.7 on the Noometry Index.

Last verified . 21 shared benchmarks.

Gemma 2 9B Google

25.7

Rank #334 Confirmed

Qwen3.5 397B-A17B Alibaba (Qwen)

45.7

Rank #60 Confirmed

Summary

  • They share 21 benchmarks with published results for both. Gemma 2 9B scores higher in 0 categories and Qwen3.5 397B-A17B in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Qwen3.5 397B-A17B leads 53.3 to 9.7.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 0.6% for Gemma 2 9B and 88.9% for Qwen3.5 397B-A17B.

Side by side

Gemma 2 9BQwen3.5 397B-A17B
ProviderGoogleAlibaba (Qwen)
Noometry Index25.745.7
Released2024-06-242026-02-01
WeightsOpenOpen
Context window—262K
Max output—66K
Input $ / M tokens—$0.60
Output $ / M tokens—$3.60
Results tracked3536

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen3.5 397B-A17B leads

Gemma 2 9B: 29.0 (#299), Qwen3.5 397B-A17B: 41.7 (#108)

BenchmarkGemma 2 9BQwen3.5 397B-A17B
LMArena Coding11741464
LMArena WebDev—1400
BigCodeBench Instruct34.7%—
LiveBench Coding22.5%—
BigCodeBench Complete40.6%—

Agentic & Tool Use Not comparable

Gemma 2 9B: —, Qwen3.5 397B-A17B: 33.2 (#51)

BenchmarkGemma 2 9BQwen3.5 397B-A17B
APEX-Agents—24.9%
τ²-bench Airline—81.5%
τ²-bench Banking—9.8%
τ²-bench Retail—84.4%
τ²-bench Telecom—97.8%

Reasoning Qwen3.5 397B-A17B leads

Gemma 2 9B: 15.3 (#306), Qwen3.5 397B-A17B: 33.6 (#62)

BenchmarkGemma 2 9BQwen3.5 397B-A17B
LMArena Hard Prompts11721448
Epoch Capabilities Index119.78146.64
Kagi LLM Benchmark—73.7%
NYT Connections (extended)—58.9%
Chess Puzzles—13%
Thematic Generalization—65.1%
LiveBench Reasoning15.2%—
Mystery Game Puzzles—18%
DTBench—87.5%
LiveBench Data Analysis36.4%—
LMCA—37.9%
LiveBench28.7%—
PIQA83.7%—

Math Qwen3.5 397B-A17B leads

Gemma 2 9B: 9.3 (#312), Qwen3.5 397B-A17B: 44.9 (#64)

BenchmarkGemma 2 9BQwen3.5 397B-A17B
OTIS Mock AIME 2024-20250.6%88.9%
LMArena Math11831455
FrontierMath (Tiers 1-3)—31.2%
LiveBench Math19.8%—
MATH Level 521%—
GSM8K84.9%—

Knowledge Qwen3.5 397B-A17B leads

Gemma 2 9B: 9.7 (#300), Qwen3.5 397B-A17B: 53.3 (#53)

BenchmarkGemma 2 9BQwen3.5 397B-A17B
GPQA Diamond27.5%86.4%
LMArena Expert11471461
BoolQ85.7%—
MMLU72.1%—

Multimodal Not comparable

Gemma 2 9B: —, Qwen3.5 397B-A17B: 40.0 (#42)

BenchmarkGemma 2 9BQwen3.5 397B-A17B
LMArena Vision—1262

Multilingual Qwen3.5 397B-A17B leads

Gemma 2 9B: 36.8 (#233), Qwen3.5 397B-A17B: 54.0 (#56)

BenchmarkGemma 2 9BQwen3.5 397B-A17B
LMArena Non-English11881431
LMArena Chinese11851501
LMArena French11891461
LMArena German11861447
LMArena Japanese11441427
LMArena Korean11371382
LMArena Russian12001429
LMArena Spanish12001443

Instruction Following Qwen3.5 397B-A17B leads

Gemma 2 9B: 57.8 (#264), Qwen3.5 397B-A17B: 75.2 (#73)

BenchmarkGemma 2 9BQwen3.5 397B-A17B
LMArena Instruction Following11781424
LiveBench Instruction Following52.6%—

Long Context Qwen3.5 397B-A17B leads

Gemma 2 9B: 36.3 (#228), Qwen3.5 397B-A17B: 44.1 (#71)

BenchmarkGemma 2 9BQwen3.5 397B-A17B
LMArena Longer Query11971442

Writing & Preference Qwen3.5 397B-A17B leads

Gemma 2 9B: 32.3 (#276), Qwen3.5 397B-A17B: 62.5 (#74)

BenchmarkGemma 2 9BQwen3.5 397B-A17B
LMArena Text12071438
LMArena Creative Writing12061401
EQ-Bench Creative Writing8411478
LMArena Multi-Turn11931447
LiveBench Language25.5%—

Frequently asked questions

Is Gemma 2 9B better than Qwen3.5 397B-A17B?

Qwen3.5 397B-A17B is the stronger model overall, scoring 45.7 to 25.7 on the Noometry Index.

Is Gemma 2 9B or Qwen3.5 397B-A17B better for coding?

Qwen3.5 397B-A17B scores higher on coding benchmarks: 41.7 versus 29.0 in the Noometry coding category.

How many benchmarks do Gemma 2 9B and Qwen3.5 397B-A17B share?

21 benchmarks have published results for both models. Gemma 2 9B has 35 scored results on Noometry and Qwen3.5 397B-A17B has 36.

Related comparisons

Go deeper