Noometry

Model comparison

GPT-4 vs Qwen Max

Qwen Max is the stronger model overall, scoring 34.7 to 29.1 on the Noometry Index.

Last verified . 20 shared benchmarks.

GPT-4 OpenAI

29.1

Rank #316 Confirmed

Qwen Max Alibaba (Qwen)

34.7

Rank #230 Confirmed

Summary

  • They share 20 benchmarks with published results for both. GPT-4 scores higher in 1 category and Qwen Max in 7 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Qwen Max leads 47.8 to 34.9.
  • The biggest single-benchmark swing is MATH Level 5: 23% for GPT-4 and 67.2% for Qwen Max.
  • Qwen Max is cheaper at $1.60 / $6.40 per million input/output tokens, against $30 / $60 for GPT-4.
  • Qwen Max accepts more context: 33K tokens versus 8K.

Side by side

GPT-4Qwen Max
ProviderOpenAIAlibaba (Qwen)
Noometry Index29.134.7
Released2023-03-142024-04-03
WeightsProprietaryProprietary
Context window8K33K
Max output8K8K
Input $ / M tokens$30$1.60
Output $ / M tokens$60$6.40
Results tracked3823

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

GPT-4: 31.6 (#283), Qwen Max: 30.7 (#292)

BenchmarkGPT-4Qwen Max
LMArena Coding12541288
Aider Polyglot—21.8%
WeirdML12.4%—
BigCodeBench Instruct46%—
BigCodeBench Complete57.2%—
HumanEval+79.3%—

Agentic & Tool Use Not comparable

GPT-4: —, Qwen Max: —

BenchmarkGPT-4Qwen Max
METR Time Horizons36.1%—

Reasoning Qwen Max leads

GPT-4: 17.8 (#289), Qwen Max: 25.1 (#151)

BenchmarkGPT-4Qwen Max
LMArena Hard Prompts12411269
Chess Puzzles4%—
Mystery Game Puzzles12%—
DTBench62.7%—
LMCA17.1%—
BIG-Bench Hard75.1%—
Epoch Capabilities Index125.89—
ForecastBench57.8—
HellaSwag95.3%—
WinoGrande87.5%—

Math Qwen Max leads

GPT-4: 10.8 (#309), Qwen Max: 22.3 (#276)

BenchmarkGPT-4Qwen Max
OTIS Mock AIME 2024-20251.1%16.1%
LMArena Math12691275
MATH Level 523%67.2%
FrontierMath (Feb 2025 set)—1%
GSM8K92%—

Knowledge Qwen Max leads

GPT-4: 18.4 (#282), Qwen Max: 30.3 (#228)

BenchmarkGPT-4Qwen Max
GPQA Diamond35.7%56.1%
LMArena Expert12111248
MMLU86.4%—
TriviaQA84.8%—

Multilingual Qwen Max leads

GPT-4: 40.6 (#215), Qwen Max: 41.8 (#202)

BenchmarkGPT-4Qwen Max
LMArena Non-English12461263
LMArena Chinese12421254
LMArena French12831330
LMArena German12511254
LMArena Japanese12091205
LMArena Korean11841142
LMArena Russian12511274
LMArena Spanish12611290

Instruction Following Qwen Max leads

GPT-4: 65.3 (#222), Qwen Max: 66.5 (#208)

BenchmarkGPT-4Qwen Max
LMArena Instruction Following12411262

Long Context Qwen Max leads

GPT-4: 37.7 (#212), Qwen Max: 39.4 (#180)

BenchmarkGPT-4Qwen Max
LMArena Longer Query12441288
Fiction.LiveBench—66.7%

Writing & Preference Qwen Max leads

GPT-4: 34.9 (#268), Qwen Max: 47.8 (#205)

BenchmarkGPT-4Qwen Max
LMArena Text12631282
LMArena Creative Writing12441248
LMArena Multi-Turn12571277
EQ-Bench Creative Writing752—

Frequently asked questions

Is GPT-4 better than Qwen Max?

Qwen Max is the stronger model overall, scoring 34.7 to 29.1 on the Noometry Index.

Which is cheaper, GPT-4 or Qwen Max?

Qwen Max is cheaper. It lists at $1.60 per million input tokens and $6.40 per million output tokens; GPT-4 lists at $30 and $60.

Is GPT-4 or Qwen Max better for coding?

They score almost the same on coding (31.6 vs 30.7); test both on your own repository before choosing.

Which has the bigger context window?

Qwen Max does, with 33K tokens against 8K.

How many benchmarks do GPT-4 and Qwen Max share?

20 benchmarks have published results for both models. GPT-4 has 38 scored results on Noometry and Qwen Max has 23.

Related comparisons

Go deeper