Noometry

Model comparison

Phi 3 Mini 4k Instruct vs Qwen2.5 Plus 1127

Qwen2.5 Plus 1127 is the stronger model overall, scoring 38.1 to 27.4 on the Noometry Index.

Last verified . 14 shared benchmarks.

Phi 3 Mini 4k Instruct Microsoft

27.4

Rank #324 Confirmed

Qwen2.5 Plus 1127 Alibaba (Qwen)

38.1

Rank #179 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Phi 3 Mini 4k Instruct scores higher in 0 categories and Qwen2.5 Plus 1127 in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Qwen2.5 Plus 1127 leads 49.5 to 27.7.
  • Phi 3 Mini 4k Instruct has downloadable open weights; the other is API-only.

Side by side

Phi 3 Mini 4k InstructQwen2.5 Plus 1127
ProviderMicrosoftAlibaba (Qwen)
Noometry Index27.438.1
Released2024-04-23—
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3514

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5 Plus 1127 leads

Phi 3 Mini 4k Instruct: 26.1 (#319), Qwen2.5 Plus 1127: 37.9 (#171)

BenchmarkPhi 3 Mini 4k InstructQwen2.5 Plus 1127
LMArena Coding10931314
LiveBench Coding15.5%—
HumanEval+59.1%—
MBPP+54.2%—

Reasoning Qwen2.5 Plus 1127 leads

Phi 3 Mini 4k Instruct: 13.6 (#326), Qwen2.5 Plus 1127: 24.8 (#139)

BenchmarkPhi 3 Mini 4k InstructQwen2.5 Plus 1127
LMArena Hard Prompts10721299
Chess Puzzles0%—
LiveBench Reasoning26.8%—
LiveBench Data Analysis34.7%—
Adversarial NLI52.8%—
BIG-Bench Hard71.7%—
HellaSwag76.7%—
LiveBench22.4%—
WinoGrande70.8%—

Math Qwen2.5 Plus 1127 leads

Phi 3 Mini 4k Instruct: 24.5 (#258), Qwen2.5 Plus 1127: 33.5 (#175)

BenchmarkPhi 3 Mini 4k InstructQwen2.5 Plus 1127
LMArena Math11111298
LiveBench Math15.7%—

Knowledge Qwen2.5 Plus 1127 leads

Phi 3 Mini 4k Instruct: 27.9 (#241), Qwen2.5 Plus 1127: 34.9 (#179)

BenchmarkPhi 3 Mini 4k InstructQwen2.5 Plus 1127
LMArena Expert10451289
ARC (AI2) Challenge84.9%—
MMLU68.8%—
OpenBookQA88%—
TriviaQA64%—

Multilingual Qwen2.5 Plus 1127 leads

Phi 3 Mini 4k Instruct: 26.5 (#275), Qwen2.5 Plus 1127: 42.1 (#196)

BenchmarkPhi 3 Mini 4k InstructQwen2.5 Plus 1127
LMArena Non-English10211265
LMArena Chinese10211314
LMArena German10441231
LMArena Japanese9351207
LMArena Russian10221271
LMArena French1075—
LMArena Korean905—
LMArena Spanish1084—

Instruction Following Qwen2.5 Plus 1127 leads

Phi 3 Mini 4k Instruct: 48.0 (#298), Qwen2.5 Plus 1127: 67.4 (#194)

BenchmarkPhi 3 Mini 4k InstructQwen2.5 Plus 1127
LMArena Instruction Following10531275
LiveBench Instruction Following39.1%—

Long Context Qwen2.5 Plus 1127 leads

Phi 3 Mini 4k Instruct: 31.7 (#271), Qwen2.5 Plus 1127: 39.2 (#180)

BenchmarkPhi 3 Mini 4k InstructQwen2.5 Plus 1127
LMArena Longer Query10441292

Writing & Preference Qwen2.5 Plus 1127 leads

Phi 3 Mini 4k Instruct: 27.7 (#295), Qwen2.5 Plus 1127: 49.5 (#186)

BenchmarkPhi 3 Mini 4k InstructQwen2.5 Plus 1127
LMArena Text10731299
LMArena Creative Writing10371262
LMArena Multi-Turn10181299
LiveBench Language9.2%—

Frequently asked questions

Is Phi 3 Mini 4k Instruct better than Qwen2.5 Plus 1127?

Qwen2.5 Plus 1127 is the stronger model overall, scoring 38.1 to 27.4 on the Noometry Index.

Is Phi 3 Mini 4k Instruct or Qwen2.5 Plus 1127 better for coding?

Qwen2.5 Plus 1127 scores higher on coding benchmarks: 37.9 versus 26.1 in the Noometry coding category.

How many benchmarks do Phi 3 Mini 4k Instruct and Qwen2.5 Plus 1127 share?

14 benchmarks have published results for both models. Phi 3 Mini 4k Instruct has 35 scored results on Noometry and Qwen2.5 Plus 1127 has 14.

Related comparisons

Go deeper