Noometry

Model comparison

Phi 3 Mini 4k Instruct vs Qwen2.5-Coder-32B

Qwen2.5-Coder-32B is the stronger model overall, scoring 33.4 to 27.9 on the Noometry Index.

Last verified . 25 shared benchmarks.

Phi 3 Mini 4k Instruct Microsoft

27.9

Rank #328 Confirmed

Qwen2.5-Coder-32B Alibaba (Qwen)

33.4

Rank #245 Confirmed

Summary

  • They share 25 benchmarks with published results for both. Phi 3 Mini 4k Instruct scores higher in 1 category and Qwen2.5-Coder-32B in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Qwen2.5-Coder-32B leads 41.6 to 27.6.
  • The biggest single-benchmark swing is LiveBench Coding: 15.5% for Phi 3 Mini 4k Instruct and 56.9% for Qwen2.5-Coder-32B.

Side by side

Phi 3 Mini 4k InstructQwen2.5-Coder-32B
ProviderMicrosoftAlibaba (Qwen)
Noometry Index27.933.4
Released2024-04-232024-09-18
WeightsOpenOpen
Context window—33K
Max output—29K
Input $ / M tokens—$0.66
Output $ / M tokens—$1
Results tracked3531

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Phi 3 Mini 4k Instruct leads

Phi 3 Mini 4k Instruct: 26.6 (#323), Qwen2.5-Coder-32B: 22.6 (#333)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Coder-32B
LiveBench Coding15.5%56.9%
LMArena Coding10931276
HumanEval+59.1%87.2%
MBPP+54.2%77%
SWE-bench Verified (bash only)—9%
Aider Polyglot—16.4%
BigCodeBench Instruct—49%
BigCodeBench Complete—58%

Reasoning Qwen2.5-Coder-32B leads

Phi 3 Mini 4k Instruct: 14.1 (#328), Qwen2.5-Coder-32B: 21.2 (#225)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Coder-32B
LiveBench Reasoning26.8%42.1%
LMArena Hard Prompts10721251
LiveBench Data Analysis34.7%49.9%
HellaSwag76.7%83%
LiveBench22.4%46.2%
WinoGrande70.8%80.8%
Chess Puzzles0%—
Adversarial NLI52.8%—
BIG-Bench Hard71.7%—
Epoch Capabilities Index—119.49

Math Qwen2.5-Coder-32B leads

Phi 3 Mini 4k Instruct: 26.6 (#257), Qwen2.5-Coder-32B: 33.3 (#204)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Coder-32B
LiveBench Math15.7%46.6%
LMArena Math11111251
GSM8K—93%

Knowledge Qwen2.5-Coder-32B leads

Phi 3 Mini 4k Instruct: 28.5 (#246), Qwen2.5-Coder-32B: 33.4 (#203)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Coder-32B
LMArena Expert10451221
ARC (AI2) Challenge84.9%70.5%
MMLU68.8%79.1%
OpenBookQA88%—
TriviaQA64%—

Multilingual Qwen2.5-Coder-32B leads

Phi 3 Mini 4k Instruct: 26.3 (#280), Qwen2.5-Coder-32B: 37.8 (#235)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Coder-32B
LMArena Non-English10211205
LMArena Chinese10211222
LMArena Russian10221228
LMArena French1076—
LMArena German1044—
LMArena Japanese935—
LMArena Korean905—
LMArena Spanish1085—

Instruction Following Qwen2.5-Coder-32B leads

Phi 3 Mini 4k Instruct: 47.7 (#303), Qwen2.5-Coder-32B: 61.4 (#245)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Coder-32B
LiveBench Instruction Following39.1%58.7%
LMArena Instruction Following10531223

Long Context Qwen2.5-Coder-32B leads

Phi 3 Mini 4k Instruct: 31.7 (#276), Qwen2.5-Coder-32B: 38.0 (#208)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Coder-32B
LMArena Longer Query10441251

Writing & Preference Qwen2.5-Coder-32B leads

Phi 3 Mini 4k Instruct: 27.6 (#300), Qwen2.5-Coder-32B: 41.6 (#240)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Coder-32B
LMArena Text10731230
LMArena Creative Writing10371174
LMArena Multi-Turn10181222
LiveBench Language9.2%23.3%

Frequently asked questions

Is Phi 3 Mini 4k Instruct better than Qwen2.5-Coder-32B?

Qwen2.5-Coder-32B is the stronger model overall, scoring 33.4 to 27.9 on the Noometry Index.

Is Phi 3 Mini 4k Instruct or Qwen2.5-Coder-32B better for coding?

Phi 3 Mini 4k Instruct scores higher on coding benchmarks: 26.6 versus 22.6 in the Noometry coding category.

How many benchmarks do Phi 3 Mini 4k Instruct and Qwen2.5-Coder-32B share?

25 benchmarks have published results for both models. Phi 3 Mini 4k Instruct has 35 scored results on Noometry and Qwen2.5-Coder-32B has 31.

Related comparisons

Go deeper