Noometry

Model comparison

Gemini 1.0 Pro vs Qwen2.5 72B Instruct

Qwen2.5 72B Instruct is the stronger model overall, scoring 31.9 to 27.3 on the Noometry Index.

Last verified . 22 shared benchmarks.

Gemini 1.0 Pro Google

27.3

Rank #332 Confirmed

Qwen2.5 72B Instruct Alibaba (Qwen)

31.9

Rank #267 Confirmed

Summary

  • They share 22 benchmarks with published results for both. Gemini 1.0 Pro scores higher in 0 categories and Qwen2.5 72B Instruct in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Qwen2.5 72B Instruct leads 27.0 to 15.6.
  • The biggest single-benchmark swing is MATH Level 5: 11.2% for Gemini 1.0 Pro and 63.2% for Qwen2.5 72B Instruct.
  • Qwen2.5 72B Instruct has downloadable open weights; the other is API-only.

Side by side

Gemini 1.0 ProQwen2.5 72B Instruct
ProviderGoogleAlibaba (Qwen)
Noometry Index27.331.9
Released2023-12-132024-09
WeightsProprietaryOpen
Context window—131K
Max output—8K
Input $ / M tokens—$1.40
Output $ / M tokens—$5.60
Results tracked2443

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5 72B Instruct leads

Gemini 1.0 Pro: 32.2 (#275), Qwen2.5 72B Instruct: 33.2 (#260)

BenchmarkGemini 1.0 ProQwen2.5 72B Instruct
LMArena Coding11081292
WeirdML—16%
BigCodeBench Instruct—45.8%
BigCodeBench Complete—55.9%
HumanEval+55.5%—
MBPP+61.4%—

Agentic & Tool Use Not comparable

Gemini 1.0 Pro: —, Qwen2.5 72B Instruct: 22.1 (#133)

BenchmarkGemini 1.0 ProQwen2.5 72B Instruct
TheAgentCompany—5.7%
BALROG—16.2%
METR Time Horizons—35.8%

Reasoning Qwen2.5 72B Instruct leads

Gemini 1.0 Pro: 17.1 (#296), Qwen2.5 72B Instruct: 22.3 (#199)

BenchmarkGemini 1.0 ProQwen2.5 72B Instruct
LMArena Hard Prompts11091271
DTBench45.9%62.9%
Epoch Capabilities Index117.04129
LMCA—13.4%
BIG-Bench Hard—79.8%
ForecastBench—57.5
HellaSwag—84.8%
PIQA—82.6%
WinoGrande—82.3%

Math Qwen2.5 72B Instruct leads

Gemini 1.0 Pro: 9.3 (#321), Qwen2.5 72B Instruct: 19.3 (#287)

BenchmarkGemini 1.0 ProQwen2.5 72B Instruct
OTIS Mock AIME 2024-20251.1%8.1%
LMArena Math11321283
MATH Level 511.2%63.2%
Omni-MATH—33%

Knowledge Qwen2.5 72B Instruct leads

Gemini 1.0 Pro: 15.6 (#291), Qwen2.5 72B Instruct: 27.0 (#253)

BenchmarkGemini 1.0 ProQwen2.5 72B Instruct
GPQA Diamond34%49.1%
LMArena Expert10591245
MMLU70%85.3%
MMLU-Pro—63.1%
Confabulations—19.1%
GPQA (HELM)—42.6%
ARC (AI2) Challenge—94.5%
TriviaQA—71.9%

Multilingual Qwen2.5 72B Instruct leads

Gemini 1.0 Pro: 33.4 (#252), Qwen2.5 72B Instruct: 41.0 (#213)

BenchmarkGemini 1.0 ProQwen2.5 72B Instruct
LMArena Non-English11381252
LMArena Chinese11241272
LMArena French11451280
LMArena German11251234
LMArena Japanese10231180
LMArena Russian11861264
LMArena Spanish11191256
LMArena Korean—1188

Instruction Following Qwen2.5 72B Instruct leads

Gemini 1.0 Pro: 57.6 (#267), Qwen2.5 72B Instruct: 65.5 (#221)

BenchmarkGemini 1.0 ProQwen2.5 72B Instruct
LMArena Instruction Following11141254
IFEval—80.6%

Long Context Qwen2.5 72B Instruct leads

Gemini 1.0 Pro: 34.3 (#249), Qwen2.5 72B Instruct: 38.9 (#188)

BenchmarkGemini 1.0 ProQwen2.5 72B Instruct
LMArena Longer Query11321282

Writing & Preference Qwen2.5 72B Instruct leads

Gemini 1.0 Pro: 36.0 (#264), Qwen2.5 72B Instruct: 46.7 (#215)

BenchmarkGemini 1.0 ProQwen2.5 72B Instruct
LMArena Text11491269
LMArena Creative Writing11311221
LMArena Multi-Turn11391272
WildBench—80.2%

Frequently asked questions

Is Gemini 1.0 Pro better than Qwen2.5 72B Instruct?

Qwen2.5 72B Instruct is the stronger model overall, scoring 31.9 to 27.3 on the Noometry Index.

Is Gemini 1.0 Pro or Qwen2.5 72B Instruct better for coding?

Qwen2.5 72B Instruct scores higher on coding benchmarks: 33.2 versus 32.2 in the Noometry coding category.

How many benchmarks do Gemini 1.0 Pro and Qwen2.5 72B Instruct share?

22 benchmarks have published results for both models. Gemini 1.0 Pro has 24 scored results on Noometry and Qwen2.5 72B Instruct has 43.

Related comparisons

Go deeper