Noometry

Model comparison

Phi 3 Mini 4k Instruct vs Qwen2.5-Max

Qwen2.5-Max is the stronger model overall, scoring 40.7 to 27.9 on the Noometry Index.

Last verified . 24 shared benchmarks.

Phi 3 Mini 4k Instruct Microsoft

27.9

Rank #328 Confirmed

Qwen2.5-Max Alibaba (Qwen)

40.7

Rank #146 Confirmed

Summary

  • They share 24 benchmarks with published results for both. Phi 3 Mini 4k Instruct scores higher in 0 categories and Qwen2.5-Max in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Qwen2.5-Max leads 55.4 to 27.6.
  • The biggest single-benchmark swing is LiveBench Coding: 15.5% for Phi 3 Mini 4k Instruct and 64.4% for Qwen2.5-Max.
  • Phi 3 Mini 4k Instruct has downloadable open weights; the other is API-only.

Side by side

Phi 3 Mini 4k InstructQwen2.5-Max
ProviderMicrosoftAlibaba (Qwen)
Noometry Index27.940.7
Released2024-04-232025-01-25
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3527

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5-Max leads

Phi 3 Mini 4k Instruct: 26.6 (#323), Qwen2.5-Max: 41.8 (#117)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Max
LiveBench Coding15.5%64.4%
LMArena Coding10931359
HumanEval+59.1%—
MBPP+54.2%—

Reasoning Qwen2.5-Max leads

Phi 3 Mini 4k Instruct: 14.1 (#328), Qwen2.5-Max: 25.6 (#147)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Max
LiveBench Reasoning26.8%51.4%
LMArena Hard Prompts10721360
LiveBench Data Analysis34.7%67.9%
LiveBench22.4%62.3%
Chess Puzzles0%—
Adversarial NLI52.8%—
BIG-Bench Hard71.7%—
Epoch Capabilities Index—132.53
HellaSwag76.7%—
WinoGrande70.8%—

Math Qwen2.5-Max leads

Phi 3 Mini 4k Instruct: 26.6 (#257), Qwen2.5-Max: 36.9 (#162)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Max
LiveBench Math15.7%58.4%
LMArena Math11111369

Knowledge Qwen2.5-Max leads

Phi 3 Mini 4k Instruct: 28.5 (#246), Qwen2.5-Max: 35.3 (#186)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Max
LMArena Expert10451337
Confabulations—21.8%
ARC (AI2) Challenge84.9%—
MMLU68.8%—
OpenBookQA88%—
TriviaQA64%—

Multilingual Qwen2.5-Max leads

Phi 3 Mini 4k Instruct: 26.3 (#280), Qwen2.5-Max: 48.1 (#146)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Max
LMArena Non-English10211352
LMArena Chinese10211382
LMArena French10761396
LMArena German10441350
LMArena Japanese9351300
LMArena Korean9051304
LMArena Russian10221353
LMArena Spanish10851377

Instruction Following Qwen2.5-Max leads

Phi 3 Mini 4k Instruct: 47.7 (#303), Qwen2.5-Max: 71.3 (#152)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Max
LiveBench Instruction Following39.1%75.3%
LMArena Instruction Following10531335

Long Context Qwen2.5-Max leads

Phi 3 Mini 4k Instruct: 31.7 (#276), Qwen2.5-Max: 41.4 (#142)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Max
LMArena Longer Query10441358

Writing & Preference Qwen2.5-Max leads

Phi 3 Mini 4k Instruct: 27.6 (#300), Qwen2.5-Max: 55.4 (#146)

BenchmarkPhi 3 Mini 4k InstructQwen2.5-Max
LMArena Text10731367
LMArena Creative Writing10371339
LMArena Multi-Turn10181364
LiveBench Language9.2%56.3%
Short-Story Creative Writing—72.9%

Frequently asked questions

Is Phi 3 Mini 4k Instruct better than Qwen2.5-Max?

Qwen2.5-Max is the stronger model overall, scoring 40.7 to 27.9 on the Noometry Index.

Is Phi 3 Mini 4k Instruct or Qwen2.5-Max better for coding?

Qwen2.5-Max scores higher on coding benchmarks: 41.8 versus 26.6 in the Noometry coding category.

How many benchmarks do Phi 3 Mini 4k Instruct and Qwen2.5-Max share?

24 benchmarks have published results for both models. Phi 3 Mini 4k Instruct has 35 scored results on Noometry and Qwen2.5-Max has 27.

Related comparisons

Go deeper