Noometry

Model comparison

Gemini 1.5 Flash 8B vs Qwen2.5-Coder-32B

Qwen2.5-Coder-32B is the stronger model overall, scoring 33.4 to 29.9 on the Noometry Index.

Last verified . 12 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.9

Rank #301 Confirmed

Qwen2.5-Coder-32B Alibaba (Qwen)

33.4

Rank #245 Confirmed

Summary

  • They share 12 benchmarks with published results for both. Gemini 1.5 Flash 8B scores higher in 4 categories and Qwen2.5-Coder-32B in 4 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Qwen2.5-Coder-32B leads 33.3 to 14.2.
  • Qwen2.5-Coder-32B has downloadable open weights; the other is API-only.

Side by side

Gemini 1.5 Flash 8BQwen2.5-Coder-32B
ProviderGoogleAlibaba (Qwen)
Noometry Index29.933.4
Released2024-10-032024-09-18
WeightsProprietaryOpen
Context window—33K
Max output—29K
Input $ / M tokens—$0.66
Output $ / M tokens—$1
Results tracked2131

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 35.5 (#225), Qwen2.5-Coder-32B: 22.6 (#333)

BenchmarkGemini 1.5 Flash 8BQwen2.5-Coder-32B
LMArena Coding12181276
SWE-bench Verified (bash only)—9%
Aider Polyglot—16.4%
BigCodeBench Instruct—49%
LiveBench Coding—56.9%
BigCodeBench Complete—58%
HumanEval+—87.2%
MBPP+—77%

Reasoning Qwen2.5-Coder-32B leads

Gemini 1.5 Flash 8B: 20.0 (#244), Qwen2.5-Coder-32B: 21.2 (#225)

BenchmarkGemini 1.5 Flash 8BQwen2.5-Coder-32B
LMArena Hard Prompts12091251
LiveBench Reasoning—42.1%
DTBench50%—
LiveBench Data Analysis—49.9%
Epoch Capabilities Index—119.49
HellaSwag—83%
LiveBench—46.2%
WinoGrande—80.8%

Math Qwen2.5-Coder-32B leads

Gemini 1.5 Flash 8B: 14.2 (#302), Qwen2.5-Coder-32B: 33.3 (#204)

BenchmarkGemini 1.5 Flash 8BQwen2.5-Coder-32B
LMArena Math12071251
OTIS Mock AIME 2024-20254.6%—
LiveBench Math—46.6%
GSM8K—93%

Knowledge Qwen2.5-Coder-32B leads

Gemini 1.5 Flash 8B: 16.0 (#289), Qwen2.5-Coder-32B: 33.4 (#203)

BenchmarkGemini 1.5 Flash 8BQwen2.5-Coder-32B
LMArena Expert11851221
GPQA Diamond33%—
ARC (AI2) Challenge—70.5%
MMLU—79.1%

Multimodal Not comparable

Gemini 1.5 Flash 8B: 28.2 (#115), Qwen2.5-Coder-32B: —

BenchmarkGemini 1.5 Flash 8BQwen2.5-Coder-32B
LMArena Vision1044—

Multilingual Too close to call

Gemini 1.5 Flash 8B: 38.5 (#229), Qwen2.5-Coder-32B: 37.8 (#235)

BenchmarkGemini 1.5 Flash 8BQwen2.5-Coder-32B
LMArena Non-English12151205
LMArena Chinese12311222
LMArena Russian12361228
LMArena French1234—
LMArena German1206—
LMArena Japanese1150—
LMArena Korean1140—
LMArena Spanish1212—

Instruction Following Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 62.8 (#236), Qwen2.5-Coder-32B: 61.4 (#245)

BenchmarkGemini 1.5 Flash 8BQwen2.5-Coder-32B
LMArena Instruction Following11991223
LiveBench Instruction Following—58.7%

Long Context Qwen2.5-Coder-32B leads

Gemini 1.5 Flash 8B: 37.0 (#225), Qwen2.5-Coder-32B: 38.0 (#208)

BenchmarkGemini 1.5 Flash 8BQwen2.5-Coder-32B
LMArena Longer Query12191251

Writing & Preference Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 42.8 (#232), Qwen2.5-Coder-32B: 41.6 (#240)

BenchmarkGemini 1.5 Flash 8BQwen2.5-Coder-32B
LMArena Text12261230
LMArena Creative Writing12181174
LMArena Multi-Turn11851222
LiveBench Language—23.3%

Frequently asked questions

Is Gemini 1.5 Flash 8B better than Qwen2.5-Coder-32B?

Qwen2.5-Coder-32B is the stronger model overall, scoring 33.4 to 29.9 on the Noometry Index.

Is Gemini 1.5 Flash 8B or Qwen2.5-Coder-32B better for coding?

Gemini 1.5 Flash 8B scores higher on coding benchmarks: 35.5 versus 22.6 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash 8B and Qwen2.5-Coder-32B share?

12 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and Qwen2.5-Coder-32B has 31.

Related comparisons

Go deeper