Noometry

Model comparison

GPT-4o mini vs Qwen2.5-Max

Qwen2.5-Max is the stronger model overall, scoring 40.1 to 25.4 on the Noometry Index.

Last verified . 27 shared benchmarks.

GPT-4o mini OpenAI

25.4

Rank #337 Confirmed

Qwen2.5-Max Alibaba (Qwen)

40.1

Rank #142 Confirmed

Summary

  • They share 27 benchmarks with published results for both. GPT-4o mini scores higher in 0 categories and Qwen2.5-Max in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Qwen2.5-Max leads 34.3 to 9.9.
  • The biggest single-benchmark swing is LiveBench Language: 28.6% for GPT-4o mini and 56.3% for Qwen2.5-Max.

Side by side

GPT-4o miniQwen2.5-Max
ProviderOpenAIAlibaba (Qwen)
Noometry Index25.440.1
Released2024-07-182025-01-25
WeightsProprietaryProprietary
Context window128K—
Max output16K—
Input $ / M tokens$0.15—
Output $ / M tokens$0.60—
Results tracked6027

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5-Max leads

GPT-4o mini: 21.6 (#330), Qwen2.5-Max: 41.2 (#118)

BenchmarkGPT-4o miniQwen2.5-Max
LiveBench Coding43.1%64.4%
LMArena Coding12901359
Aider Polyglot3.6%—
WeirdML11.8%—
BigCodeBench Instruct46.1%—
BigCodeBench Complete57.4%—
HumanEval+83.5%—
MBPP+72.2%—

Agentic & Tool Use Not comparable

GPT-4o mini: 27.2 (#95), Qwen2.5-Max: —

BenchmarkGPT-4o miniQwen2.5-Max
BALROG17.4%—

Reasoning Qwen2.5-Max leads

GPT-4o mini: 8.7 (#343), Qwen2.5-Max: 24.6 (#144)

BenchmarkGPT-4o miniQwen2.5-Max
LiveBench Reasoning32.8%51.4%
LMArena Hard Prompts12671360
LiveBench Data Analysis50%67.9%
Epoch Capabilities Index126.56132.54
LiveBench41.3%62.3%
ARC-AGI-20%—
SimpleBench10.7%—
Kagi LLM Benchmark28.8%—
Chess Puzzles0%—
Mystery Game Puzzles12%—
DTBench54.4%—
LMCA10.4%—
PIQA88.7%—

Math Qwen2.5-Max leads

GPT-4o mini: 9.9 (#307), Qwen2.5-Max: 34.3 (#162)

BenchmarkGPT-4o miniQwen2.5-Max
LiveBench Math36.3%58.4%
LMArena Math12671369
FrontierMath (Tiers 1-3)0.7%—
OTIS Mock AIME 2024-20256.9%—
Omni-MATH28%—
MATH Level 552.6%—
GSM8K91.3%—

Knowledge Qwen2.5-Max leads

GPT-4o mini: 17.7 (#279), Qwen2.5-Max: 34.8 (#181)

BenchmarkGPT-4o miniQwen2.5-Max
Confabulations37.2%21.8%
LMArena Expert12351337
GPQA Diamond37.7%—
SimpleQA Verified8.3%—
MMLU-Pro60.3%—
GPQA (HELM)36.8%—
BoolQ88.7%—
MMLU81.8%—

Multimodal Not comparable

GPT-4o mini: 25.3 (#117), Qwen2.5-Max: —

BenchmarkGPT-4o miniQwen2.5-Max
LMArena Vision1066—
Video-MME64.8%—
GeoBench64%—
VPCT34%—

Multilingual Qwen2.5-Max leads

GPT-4o mini: 42.2 (#194), Qwen2.5-Max: 48.3 (#140)

BenchmarkGPT-4o miniQwen2.5-Max
LMArena Non-English12661352
LMArena Chinese12641382
LMArena French12961396
LMArena German12721350
LMArena Japanese12161299
LMArena Korean11951304
LMArena Russian12751353
LMArena Spanish12761377

Instruction Following Qwen2.5-Max leads

GPT-4o mini: 62.1 (#234), Qwen2.5-Max: 71.5 (#147)

BenchmarkGPT-4o miniQwen2.5-Max
LiveBench Instruction Following56.8%75.3%
LMArena Instruction Following12591335
IFEval78.2%—

Long Context Qwen2.5-Max leads

GPT-4o mini: 39.1 (#181), Qwen2.5-Max: 41.3 (#137)

BenchmarkGPT-4o miniQwen2.5-Max
LMArena Longer Query12891358

Writing & Preference Qwen2.5-Max leads

GPT-4o mini: 39.7 (#243), Qwen2.5-Max: 55.6 (#141)

BenchmarkGPT-4o miniQwen2.5-Max
LMArena Text12861367
LMArena Creative Writing12681339
Short-Story Creative Writing67.2%72.9%
LMArena Multi-Turn12851364
LiveBench Language28.6%56.3%
EQ-Bench Creative Writing873—
WildBench79.1%—

Frequently asked questions

Is GPT-4o mini better than Qwen2.5-Max?

Qwen2.5-Max is the stronger model overall, scoring 40.1 to 25.4 on the Noometry Index.

Is GPT-4o mini or Qwen2.5-Max better for coding?

Qwen2.5-Max scores higher on coding benchmarks: 41.2 versus 21.6 in the Noometry coding category.

How many benchmarks do GPT-4o mini and Qwen2.5-Max share?

27 benchmarks have published results for both models. GPT-4o mini has 60 scored results on Noometry and Qwen2.5-Max has 27.

Related comparisons

Go deeper