Noometry

Model comparison

Phi 3 Small 8k Instruct vs Qwen2.5-Max

Qwen2.5-Max is the stronger model overall, scoring 40.7 to 29.3 on the Noometry Index.

Last verified . 24 shared benchmarks.

Phi 3 Small 8k Instruct Microsoft

29.3

Rank #314 Confirmed

Qwen2.5-Max Alibaba (Qwen)

40.7

Rank #146 Confirmed

Summary

  • They share 24 benchmarks with published results for both. Phi 3 Small 8k Instruct scores higher in 0 categories and Qwen2.5-Max in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Qwen2.5-Max leads 55.4 to 31.1.
  • The biggest single-benchmark swing is LiveBench Coding: 20.3% for Phi 3 Small 8k Instruct and 64.4% for Qwen2.5-Max.
  • Phi 3 Small 8k Instruct has downloadable open weights; the other is API-only.

Side by side

Phi 3 Small 8k InstructQwen2.5-Max
ProviderMicrosoftAlibaba (Qwen)
Noometry Index29.340.7
Released2024-04-232025-01-25
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3227

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5-Max leads

Phi 3 Small 8k Instruct: 27.9 (#318), Qwen2.5-Max: 41.8 (#117)

BenchmarkPhi 3 Small 8k InstructQwen2.5-Max
LiveBench Coding20.3%64.4%
LMArena Coding11011359

Reasoning Qwen2.5-Max leads

Phi 3 Small 8k Instruct: 14.8 (#323), Qwen2.5-Max: 25.6 (#147)

BenchmarkPhi 3 Small 8k InstructQwen2.5-Max
LiveBench Reasoning15.9%51.4%
LMArena Hard Prompts11001360
LiveBench Data Analysis30.3%67.9%
LiveBench24%62.3%
Adversarial NLI58.1%—
BIG-Bench Hard79.1%—
Epoch Capabilities Index—132.53
HellaSwag77%—
WinoGrande81.5%—

Math Qwen2.5-Max leads

Phi 3 Small 8k Instruct: 27.6 (#248), Qwen2.5-Max: 36.9 (#162)

BenchmarkPhi 3 Small 8k InstructQwen2.5-Max
LiveBench Math17.6%58.4%
LMArena Math11511369

Knowledge Qwen2.5-Max leads

Phi 3 Small 8k Instruct: 29.1 (#240), Qwen2.5-Max: 35.3 (#186)

BenchmarkPhi 3 Small 8k InstructQwen2.5-Max
LMArena Expert10671337
Confabulations—21.8%
ARC (AI2) Challenge90.7%—
MMLU75.7%—
OpenBookQA88%—
TriviaQA58.1%—

Multilingual Qwen2.5-Max leads

Phi 3 Small 8k Instruct: 28.5 (#272), Qwen2.5-Max: 48.1 (#146)

BenchmarkPhi 3 Small 8k InstructQwen2.5-Max
LMArena Non-English10581352
LMArena Chinese10611382
LMArena French11351396
LMArena German10801350
LMArena Japanese9661300
LMArena Korean8941304
LMArena Russian11111353
LMArena Spanish11111377

Instruction Following Qwen2.5-Max leads

Phi 3 Small 8k Instruct: 51.9 (#292), Qwen2.5-Max: 71.3 (#152)

BenchmarkPhi 3 Small 8k InstructQwen2.5-Max
LiveBench Instruction Following47.2%75.3%
LMArena Instruction Following10871335

Long Context Qwen2.5-Max leads

Phi 3 Small 8k Instruct: 33.0 (#267), Qwen2.5-Max: 41.4 (#142)

BenchmarkPhi 3 Small 8k InstructQwen2.5-Max
LMArena Longer Query10881358

Writing & Preference Qwen2.5-Max leads

Phi 3 Small 8k Instruct: 31.1 (#284), Qwen2.5-Max: 55.4 (#146)

BenchmarkPhi 3 Small 8k InstructQwen2.5-Max
LMArena Text11101367
LMArena Creative Writing10831339
LMArena Multi-Turn10681364
LiveBench Language12.9%56.3%
Short-Story Creative Writing—72.9%

Frequently asked questions

Is Phi 3 Small 8k Instruct better than Qwen2.5-Max?

Qwen2.5-Max is the stronger model overall, scoring 40.7 to 29.3 on the Noometry Index.

Is Phi 3 Small 8k Instruct or Qwen2.5-Max better for coding?

Qwen2.5-Max scores higher on coding benchmarks: 41.8 versus 27.9 in the Noometry coding category.

How many benchmarks do Phi 3 Small 8k Instruct and Qwen2.5-Max share?

24 benchmarks have published results for both models. Phi 3 Small 8k Instruct has 32 scored results on Noometry and Qwen2.5-Max has 27.

Related comparisons

Go deeper