Noometry

Model comparison

Phi 3 Mini 128k Instruct vs Qwen2.5-Coder-32B

Qwen2.5-Coder-32B is the stronger model overall, scoring 33.4 to 29.7 on the Noometry Index.

Last verified . 14 shared benchmarks.

Phi 3 Mini 128k Instruct Microsoft

29.7

Rank #305 Confirmed

Qwen2.5-Coder-32B Alibaba (Qwen)

33.4

Rank #245 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Phi 3 Mini 128k Instruct scores higher in 1 category and Qwen2.5-Coder-32B in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Qwen2.5-Coder-32B leads 41.6 to 27.1.
  • The biggest single-benchmark swing is BigCodeBench Instruct: 29.6% for Phi 3 Mini 128k Instruct and 49% for Qwen2.5-Coder-32B.

Side by side

Phi 3 Mini 128k InstructQwen2.5-Coder-32B
ProviderMicrosoftAlibaba (Qwen)
Noometry Index29.733.4
Released2024-04-232024-09-18
WeightsOpenOpen
Context window—33K
Max output—29K
Input $ / M tokens—$0.66
Output $ / M tokens—$1
Results tracked1931

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Phi 3 Mini 128k Instruct leads

Phi 3 Mini 128k Instruct: 28.8 (#312), Qwen2.5-Coder-32B: 22.6 (#333)

BenchmarkPhi 3 Mini 128k InstructQwen2.5-Coder-32B
BigCodeBench Instruct29.6%49%
LMArena Coding10391276
BigCodeBench Complete40.6%58%
SWE-bench Verified (bash only)—9%
Aider Polyglot—16.4%
LiveBench Coding—56.9%
HumanEval+—87.2%
MBPP+—77%

Reasoning Qwen2.5-Coder-32B leads

Phi 3 Mini 128k Instruct: 19.6 (#256), Qwen2.5-Coder-32B: 21.2 (#225)

BenchmarkPhi 3 Mini 128k InstructQwen2.5-Coder-32B
LMArena Hard Prompts10281251
LiveBench Reasoning—42.1%
LiveBench Data Analysis—49.9%
Epoch Capabilities Index—119.49
HellaSwag—83%
LiveBench—46.2%
WinoGrande—80.8%

Math Qwen2.5-Coder-32B leads

Phi 3 Mini 128k Instruct: 31.6 (#222), Qwen2.5-Coder-32B: 33.3 (#204)

BenchmarkPhi 3 Mini 128k InstructQwen2.5-Coder-32B
LMArena Math10891251
LiveBench Math—46.6%
GSM8K—93%

Knowledge Qwen2.5-Coder-32B leads

Phi 3 Mini 128k Instruct: 26.8 (#254), Qwen2.5-Coder-32B: 33.4 (#203)

BenchmarkPhi 3 Mini 128k InstructQwen2.5-Coder-32B
LMArena Expert9841221
ARC (AI2) Challenge—70.5%
MMLU—79.1%

Multilingual Qwen2.5-Coder-32B leads

Phi 3 Mini 128k Instruct: 25.2 (#285), Qwen2.5-Coder-32B: 37.8 (#235)

BenchmarkPhi 3 Mini 128k InstructQwen2.5-Coder-32B
LMArena Non-English10001205
LMArena Chinese10161222
LMArena Russian10041228
LMArena French1039—
LMArena German1006—
LMArena Japanese899—
LMArena Korean856—
LMArena Spanish1059—

Instruction Following Qwen2.5-Coder-32B leads

Phi 3 Mini 128k Instruct: 51.8 (#294), Qwen2.5-Coder-32B: 61.4 (#245)

BenchmarkPhi 3 Mini 128k InstructQwen2.5-Coder-32B
LMArena Instruction Following10231223
LiveBench Instruction Following—58.7%

Long Context Qwen2.5-Coder-32B leads

Phi 3 Mini 128k Instruct: 30.4 (#289), Qwen2.5-Coder-32B: 38.0 (#208)

BenchmarkPhi 3 Mini 128k InstructQwen2.5-Coder-32B
LMArena Longer Query9961251

Writing & Preference Qwen2.5-Coder-32B leads

Phi 3 Mini 128k Instruct: 27.1 (#301), Qwen2.5-Coder-32B: 41.6 (#240)

BenchmarkPhi 3 Mini 128k InstructQwen2.5-Coder-32B
LMArena Text10501230
LMArena Creative Writing10241174
LMArena Multi-Turn9891222
LiveBench Language—23.3%

Frequently asked questions

Is Phi 3 Mini 128k Instruct better than Qwen2.5-Coder-32B?

Qwen2.5-Coder-32B is the stronger model overall, scoring 33.4 to 29.7 on the Noometry Index.

Is Phi 3 Mini 128k Instruct or Qwen2.5-Coder-32B better for coding?

Phi 3 Mini 128k Instruct scores higher on coding benchmarks: 28.8 versus 22.6 in the Noometry coding category.

How many benchmarks do Phi 3 Mini 128k Instruct and Qwen2.5-Coder-32B share?

14 benchmarks have published results for both models. Phi 3 Mini 128k Instruct has 19 scored results on Noometry and Qwen2.5-Coder-32B has 31.

Related comparisons

Go deeper