Noometry

Model comparison

Gemma 2B vs Qwen3.5 397B-A17B

Qwen3.5 397B-A17B is the stronger model overall, scoring 45.7 to 29.1 on the Noometry Index.

Last verified . 12 shared benchmarks.

Gemma 2B Google

29.1

Rank #305 Confirmed

Qwen3.5 397B-A17B Alibaba (Qwen)

45.7

Rank #60 Confirmed

Summary

  • They share 12 benchmarks with published results for both. Gemma 2B scores higher in 0 categories and Qwen3.5 397B-A17B in 7 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Qwen3.5 397B-A17B leads 62.5 to 24.1.

Side by side

Gemma 2BQwen3.5 397B-A17B
ProviderGoogleAlibaba (Qwen)
Noometry Index29.145.7
Released2024-02-212026-02-01
WeightsOpenOpen
Context window—262K
Max output—66K
Input $ / M tokens—$0.60
Output $ / M tokens—$3.60
Results tracked2336

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen3.5 397B-A17B leads

Gemma 2B: 28.9 (#300), Qwen3.5 397B-A17B: 41.7 (#108)

BenchmarkGemma 2BQwen3.5 397B-A17B
LMArena Coding10101464
LMArena WebDev—1400
HumanEval+20.7%—
MBPP+34.1%—

Agentic & Tool Use Not comparable

Gemma 2B: —, Qwen3.5 397B-A17B: 33.2 (#51)

BenchmarkGemma 2BQwen3.5 397B-A17B
APEX-Agents—24.9%
τ²-bench Airline—81.5%
τ²-bench Banking—9.8%
τ²-bench Retail—84.4%
τ²-bench Telecom—97.8%

Reasoning Qwen3.5 397B-A17B leads

Gemma 2B: 17.9 (#275), Qwen3.5 397B-A17B: 33.6 (#62)

BenchmarkGemma 2BQwen3.5 397B-A17B
LMArena Hard Prompts9891448
Epoch Capabilities Index93.57146.64
Kagi LLM Benchmark—73.7%
NYT Connections (extended)—58.9%
Chess Puzzles—13%
Thematic Generalization—65.1%
Mystery Game Puzzles—18%
DTBench—87.5%
LMCA—37.9%
BIG-Bench Hard35.2%—
HellaSwag71.4%—
PIQA77.3%—
WinoGrande65.4%—

Math Qwen3.5 397B-A17B leads

Gemma 2B: 27.6 (#237), Qwen3.5 397B-A17B: 44.9 (#64)

BenchmarkGemma 2BQwen3.5 397B-A17B
LMArena Math10091455
FrontierMath (Tiers 1-3)—31.2%
OTIS Mock AIME 2024-2025—88.9%
GSM8K17.7%—

Knowledge Not comparable

Gemma 2B: —, Qwen3.5 397B-A17B: 53.3 (#53)

BenchmarkGemma 2BQwen3.5 397B-A17B
GPQA Diamond—86.4%
LMArena Expert—1461
ARC (AI2) Challenge42.1%—
BoolQ69.4%—
MMLU42.3%—
TriviaQA53.2%—

Multimodal Not comparable

Gemma 2B: —, Qwen3.5 397B-A17B: 40.0 (#42)

BenchmarkGemma 2BQwen3.5 397B-A17B
LMArena Vision—1262

Multilingual Qwen3.5 397B-A17B leads

Gemma 2B: 23.1 (#289), Qwen3.5 397B-A17B: 54.0 (#56)

BenchmarkGemma 2BQwen3.5 397B-A17B
LMArena Non-English9581431
LMArena Chinese9861501
LMArena Russian9371429
LMArena French—1461
LMArena German—1447
LMArena Japanese—1427
LMArena Korean—1382
LMArena Spanish—1443

Instruction Following Qwen3.5 397B-A17B leads

Gemma 2B: 48.8 (#297), Qwen3.5 397B-A17B: 75.2 (#73)

BenchmarkGemma 2BQwen3.5 397B-A17B
LMArena Instruction Following9711424

Long Context Qwen3.5 397B-A17B leads

Gemma 2B: 29.9 (#286), Qwen3.5 397B-A17B: 44.1 (#71)

BenchmarkGemma 2BQwen3.5 397B-A17B
LMArena Longer Query9811442

Writing & Preference Qwen3.5 397B-A17B leads

Gemma 2B: 24.1 (#303), Qwen3.5 397B-A17B: 62.5 (#74)

BenchmarkGemma 2BQwen3.5 397B-A17B
LMArena Text10021438
LMArena Creative Writing9861401
LMArena Multi-Turn9451447
EQ-Bench Creative Writing—1478

Frequently asked questions

Is Gemma 2B better than Qwen3.5 397B-A17B?

Qwen3.5 397B-A17B is the stronger model overall, scoring 45.7 to 29.1 on the Noometry Index.

Is Gemma 2B or Qwen3.5 397B-A17B better for coding?

Qwen3.5 397B-A17B scores higher on coding benchmarks: 41.7 versus 28.9 in the Noometry coding category.

How many benchmarks do Gemma 2B and Qwen3.5 397B-A17B share?

12 benchmarks have published results for both models. Gemma 2B has 23 scored results on Noometry and Qwen3.5 397B-A17B has 36.

Related comparisons

Go deeper