Noometry

Model comparison

Phi-4 vs Qwen Max

Qwen Max is the stronger model overall, scoring 34.7 to 31.2 on the Noometry Index. Phi-4 costs 32× less per token, which makes it the better buy when Qwen Max's lead doesn't matter for your workload.

Last verified . 20 shared benchmarks.

Phi-4 Microsoft

31.2

Rank #279 Confirmed

Qwen Max Alibaba (Qwen)

34.7

Rank #230 Confirmed

Summary

  • They share 20 benchmarks with published results for both. Phi-4 scores higher in 2 categories and Qwen Max in 6 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Qwen Max leads 25.1 to 17.7.
  • Phi-4 is cheaper at $0.07 / $0.14 per million input/output tokens, against $1.60 / $6.40 for Qwen Max.
  • Phi-4 accepts more context: 128K tokens versus 33K.
  • Phi-4 has downloadable open weights; the other is API-only.

Side by side

Phi-4Qwen Max
ProviderMicrosoftAlibaba (Qwen)
Noometry Index31.234.7
Released2024-12-112024-04-03
WeightsOpenProprietary
Context window128K33K
Max output4K8K
Input $ / M tokens$0.07$1.60
Output $ / M tokens$0.14$6.40
Results tracked3723

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Phi-4 leads

Phi-4: 34.4 (#239), Qwen Max: 30.7 (#292)

BenchmarkPhi-4Qwen Max
LMArena Coding12311288
Aider Polyglot—21.8%
BigCodeBench Instruct45.5%—
LiveBench Coding30.7%—
BigCodeBench Complete55.4%—

Agentic & Tool Use Not comparable

Phi-4: 22.8 (#128), Qwen Max: —

BenchmarkPhi-4Qwen Max
Berkeley Function Calling Leaderboard28.8%—
BALROG11.6%—

Reasoning Qwen Max leads

Phi-4: 17.7 (#291), Qwen Max: 25.1 (#151)

BenchmarkPhi-4Qwen Max
LMArena Hard Prompts12201269
Chess Puzzles1%—
LiveBench Reasoning47.8%—
LiveBench Data Analysis45.2%—
Epoch Capabilities Index130.42—
LiveBench41.6%—

Math Qwen Max leads

Phi-4: 20.8 (#285), Qwen Max: 22.3 (#276)

BenchmarkPhi-4Qwen Max
OTIS Mock AIME 2024-202513.8%16.1%
LMArena Math12461275
MATH Level 564.9%67.2%
LiveBench Math42%—
FrontierMath (Feb 2025 set)—1%

Knowledge Phi-4 leads

Phi-4: 32.6 (#209), Qwen Max: 30.3 (#228)

BenchmarkPhi-4Qwen Max
GPQA Diamond56.1%56.1%
LMArena Expert12031248
Confabulations29.4%—
Vectara Hallucination Rate3.7%—
MMLU84.8%—

Multilingual Qwen Max leads

Phi-4: 37.2 (#237), Qwen Max: 41.8 (#202)

BenchmarkPhi-4Qwen Max
LMArena Non-English11971263
LMArena Chinese12121254
LMArena French12241330
LMArena German12221254
LMArena Japanese11581205
LMArena Korean11511142
LMArena Russian12091274
LMArena Spanish12341290

Instruction Following Qwen Max leads

Phi-4: 60.4 (#251), Qwen Max: 66.5 (#208)

BenchmarkPhi-4Qwen Max
LMArena Instruction Following12011262
LiveBench Instruction Following58.4%—

Long Context Qwen Max leads

Phi-4: 36.9 (#226), Qwen Max: 39.4 (#180)

BenchmarkPhi-4Qwen Max
LMArena Longer Query12171288
Fiction.LiveBench—66.7%

Writing & Preference Qwen Max leads

Phi-4: 40.5 (#244), Qwen Max: 47.8 (#205)

BenchmarkPhi-4Qwen Max
LMArena Text12171282
LMArena Creative Writing11821248
LMArena Multi-Turn12061277
Short-Story Creative Writing62.6%—
LiveBench Language25.6%—

Frequently asked questions

Is Phi-4 better than Qwen Max?

Qwen Max is the stronger model overall, scoring 34.7 to 31.2 on the Noometry Index. Phi-4 costs 32× less per token, which makes it the better buy when Qwen Max's lead doesn't matter for your workload.

Which is cheaper, Phi-4 or Qwen Max?

Phi-4 is cheaper. It lists at $0.07 per million input tokens and $0.14 per million output tokens; Qwen Max lists at $1.60 and $6.40.

Is Phi-4 or Qwen Max better for coding?

Phi-4 scores higher on coding benchmarks: 34.4 versus 30.7 in the Noometry coding category.

Which has the bigger context window?

Phi-4 does, with 128K tokens against 33K.

How many benchmarks do Phi-4 and Qwen Max share?

20 benchmarks have published results for both models. Phi-4 has 37 scored results on Noometry and Qwen Max has 23.

Related comparisons

Go deeper