Noometry

Model comparison

Gemini 1.0 Pro vs Qwen2.5-Coder-32B

Qwen2.5-Coder-32B is the stronger model overall, scoring 33.4 to 27.3 on the Noometry Index.

Last verified . 16 shared benchmarks.

Gemini 1.0 Pro Google

27.3

Rank #332 Confirmed

Qwen2.5-Coder-32B Alibaba (Qwen)

33.4

Rank #245 Confirmed

Summary

  • They share 16 benchmarks with published results for both. Gemini 1.0 Pro scores higher in 1 category and Qwen2.5-Coder-32B in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Qwen2.5-Coder-32B leads 33.3 to 9.3.
  • Qwen2.5-Coder-32B has downloadable open weights; the other is API-only.

Side by side

Gemini 1.0 ProQwen2.5-Coder-32B
ProviderGoogleAlibaba (Qwen)
Noometry Index27.333.4
Released2023-12-132024-09-18
WeightsProprietaryOpen
Context window—33K
Max output—29K
Input $ / M tokens—$0.66
Output $ / M tokens—$1
Results tracked2431

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 1.0 Pro leads

Gemini 1.0 Pro: 32.2 (#275), Qwen2.5-Coder-32B: 22.6 (#333)

BenchmarkGemini 1.0 ProQwen2.5-Coder-32B
LMArena Coding11081276
HumanEval+55.5%87.2%
MBPP+61.4%77%
SWE-bench Verified (bash only)—9%
Aider Polyglot—16.4%
BigCodeBench Instruct—49%
LiveBench Coding—56.9%
BigCodeBench Complete—58%

Reasoning Qwen2.5-Coder-32B leads

Gemini 1.0 Pro: 17.1 (#296), Qwen2.5-Coder-32B: 21.2 (#225)

BenchmarkGemini 1.0 ProQwen2.5-Coder-32B
LMArena Hard Prompts11091251
Epoch Capabilities Index117.04119.49
LiveBench Reasoning—42.1%
DTBench45.9%—
LiveBench Data Analysis—49.9%
HellaSwag—83%
LiveBench—46.2%
WinoGrande—80.8%

Math Qwen2.5-Coder-32B leads

Gemini 1.0 Pro: 9.3 (#321), Qwen2.5-Coder-32B: 33.3 (#204)

BenchmarkGemini 1.0 ProQwen2.5-Coder-32B
LMArena Math11321251
OTIS Mock AIME 2024-20251.1%—
LiveBench Math—46.6%
MATH Level 511.2%—
GSM8K—93%

Knowledge Qwen2.5-Coder-32B leads

Gemini 1.0 Pro: 15.6 (#291), Qwen2.5-Coder-32B: 33.4 (#203)

BenchmarkGemini 1.0 ProQwen2.5-Coder-32B
LMArena Expert10591221
MMLU70%79.1%
GPQA Diamond34%—
ARC (AI2) Challenge—70.5%

Multilingual Qwen2.5-Coder-32B leads

Gemini 1.0 Pro: 33.4 (#252), Qwen2.5-Coder-32B: 37.8 (#235)

BenchmarkGemini 1.0 ProQwen2.5-Coder-32B
LMArena Non-English11381205
LMArena Chinese11241222
LMArena Russian11861228
LMArena French1145—
LMArena German1125—
LMArena Japanese1023—
LMArena Spanish1119—

Instruction Following Qwen2.5-Coder-32B leads

Gemini 1.0 Pro: 57.6 (#267), Qwen2.5-Coder-32B: 61.4 (#245)

BenchmarkGemini 1.0 ProQwen2.5-Coder-32B
LMArena Instruction Following11141223
LiveBench Instruction Following—58.7%

Long Context Qwen2.5-Coder-32B leads

Gemini 1.0 Pro: 34.3 (#249), Qwen2.5-Coder-32B: 38.0 (#208)

BenchmarkGemini 1.0 ProQwen2.5-Coder-32B
LMArena Longer Query11321251

Writing & Preference Qwen2.5-Coder-32B leads

Gemini 1.0 Pro: 36.0 (#264), Qwen2.5-Coder-32B: 41.6 (#240)

BenchmarkGemini 1.0 ProQwen2.5-Coder-32B
LMArena Text11491230
LMArena Creative Writing11311174
LMArena Multi-Turn11391222
LiveBench Language—23.3%

Frequently asked questions

Is Gemini 1.0 Pro better than Qwen2.5-Coder-32B?

Qwen2.5-Coder-32B is the stronger model overall, scoring 33.4 to 27.3 on the Noometry Index.

Is Gemini 1.0 Pro or Qwen2.5-Coder-32B better for coding?

Gemini 1.0 Pro scores higher on coding benchmarks: 32.2 versus 22.6 in the Noometry coding category.

How many benchmarks do Gemini 1.0 Pro and Qwen2.5-Coder-32B share?

16 benchmarks have published results for both models. Gemini 1.0 Pro has 24 scored results on Noometry and Qwen2.5-Coder-32B has 31.

Related comparisons

Go deeper