Noometry

Model comparison

Gemma 2 27B vs Qwen Plus

Qwen Plus is the stronger model overall, scoring 37.1 to 29.4 on the Noometry Index.

Last verified . 18 shared benchmarks.

Gemma 2 27B Google

29.4

Rank #312 Confirmed

Qwen Plus Alibaba (Qwen)

37.1

Rank #210 Confirmed

Summary

  • They share 18 benchmarks with published results for both. Gemma 2 27B scores higher in 0 categories and Qwen Plus in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Qwen Plus leads 28.4 to 15.3.
  • The biggest single-benchmark swing is MATH Level 5: 27.9% for Gemma 2 27B and 65.3% for Qwen Plus.
  • Qwen Plus is cheaper at $0.40 / $1.20 per million input/output tokens, against $0.65 / $0.65 for Gemma 2 27B.
  • Qwen Plus accepts more context: 1M tokens versus 8K.
  • Gemma 2 27B has downloadable open weights; the other is API-only.

Side by side

Gemma 2 27BQwen Plus
ProviderGoogleAlibaba (Qwen)
Noometry Index29.437.1
Released2024-06-242024-01-25
WeightsOpenProprietary
Context window8K1M
Max output2K33K
Input $ / M tokens$0.65$0.40
Output $ / M tokens$0.65$1.20
Results tracked3420

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen Plus leads

Gemma 2 27B: 34.1 (#246), Qwen Plus: 38.9 (#167)

BenchmarkGemma 2 27BQwen Plus
LMArena Coding12111328
BigCodeBench Instruct42.8%—
LiveBench Coding36%—
BigCodeBench Complete52.5%—

Reasoning Qwen Plus leads

Gemma 2 27B: 15.3 (#315), Qwen Plus: 28.4 (#107)

BenchmarkGemma 2 27BQwen Plus
LMArena Hard Prompts11981317
DTBench48%81.1%
LMCA7.1%24%
Kagi LLM Benchmark—63.3%
LiveBench Reasoning28.1%—
LiveBench Data Analysis47.9%—
Epoch Capabilities Index122.08—
LiveBench38.2%—

Math Qwen Plus leads

Gemma 2 27B: 10.7 (#311), Qwen Plus: 23.3 (#271)

BenchmarkGemma 2 27BQwen Plus
OTIS Mock AIME 2024-20251.4%17.8%
LMArena Math12121326
MATH Level 527.9%65.3%
LiveBench Math26.5%—
FrontierMath (Feb 2025 set)—1.7%

Knowledge Qwen Plus leads

Gemma 2 27B: 19.0 (#280), Qwen Plus: 27.4 (#251)

BenchmarkGemma 2 27BQwen Plus
GPQA Diamond36.5%48.1%
LMArena Expert11721328
Confabulations27.1%—
MMLU75.7%—

Multilingual Qwen Plus leads

Gemma 2 27B: 38.6 (#226), Qwen Plus: 45.1 (#175)

BenchmarkGemma 2 27BQwen Plus
LMArena Non-English12171310
LMArena Chinese12211347
LMArena Japanese11751251
LMArena Russian12341323
LMArena French1247—
LMArena German1209—
LMArena Korean1174—
LMArena Spanish1228—

Instruction Following Qwen Plus leads

Gemma 2 27B: 60.5 (#249), Qwen Plus: 68.8 (#181)

BenchmarkGemma 2 27BQwen Plus
LMArena Instruction Following12061303
LiveBench Instruction Following58.1%—

Long Context Qwen Plus leads

Gemma 2 27B: 37.3 (#218), Qwen Plus: 40.3 (#158)

BenchmarkGemma 2 27BQwen Plus
LMArena Longer Query12311324

Writing & Preference Qwen Plus leads

Gemma 2 27B: 44.2 (#225), Qwen Plus: 52.2 (#176)

BenchmarkGemma 2 27BQwen Plus
LMArena Text12311326
LMArena Creative Writing12411293
LMArena Multi-Turn12241336
LiveBench Language32.6%—

Frequently asked questions

Is Gemma 2 27B better than Qwen Plus?

Qwen Plus is the stronger model overall, scoring 37.1 to 29.4 on the Noometry Index.

Which is cheaper, Gemma 2 27B or Qwen Plus?

Qwen Plus is cheaper. It lists at $0.40 per million input tokens and $1.20 per million output tokens; Gemma 2 27B lists at $0.65 and $0.65.

Is Gemma 2 27B or Qwen Plus better for coding?

Qwen Plus scores higher on coding benchmarks: 38.9 versus 34.1 in the Noometry coding category.

Which has the bigger context window?

Qwen Plus does, with 1M tokens against 8K.

How many benchmarks do Gemma 2 27B and Qwen Plus share?

18 benchmarks have published results for both models. Gemma 2 27B has 34 scored results on Noometry and Qwen Plus has 20.

Related comparisons

Go deeper