Noometry

Model comparison

GPT-4 Turbo vs Qwen2.5 14B Instruct

GPT-4 Turbo has enough public results to be ranked (#292); Qwen2.5 14B Instruct does not yet, so treat this comparison as directional.

Last verified . 3 shared benchmarks.

GPT-4 Turbo OpenAI

30.5

Rank #292 Confirmed

Qwen2.5 14B Instruct Alibaba (Qwen)

38.7

Unranked Sparse

Summary

  • They share 3 benchmarks with published results for both. GPT-4 Turbo scores higher in 0 categories and Qwen2.5 14B Instruct in 1 category; one gap is clear of the uncertainty.
  • The widest gap is in coding, where Qwen2.5 14B Instruct leads 37.7 to 33.8.
  • The biggest single-benchmark swing is BigCodeBench Instruct: 48.2% for GPT-4 Turbo and 39.8% for Qwen2.5 14B Instruct.
  • Qwen2.5 14B Instruct is cheaper at $0.35 / $1.40 per million input/output tokens, against $10 / $30 for GPT-4 Turbo.
  • Qwen2.5 14B Instruct accepts more context: 131K tokens versus 128K.
  • Qwen2.5 14B Instruct has downloadable open weights; the other is API-only.

Side by side

GPT-4 TurboQwen2.5 14B Instruct
ProviderOpenAIAlibaba (Qwen)
Noometry Index30.538.7
Released2023-11-062024-09
WeightsProprietaryOpen
Context window128K131K
Max output4K8K
Input $ / M tokens$10$0.35
Output $ / M tokens$30$1.40
Results tracked363

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5 14B Instruct leads

GPT-4 Turbo: 33.8 (#249), Qwen2.5 14B Instruct: 37.7 (#191)

BenchmarkGPT-4 TurboQwen2.5 14B Instruct
BigCodeBench Instruct48.2%39.8%
BigCodeBench Complete58.2%52.2%
WeirdML18%—
LMArena Coding1268—
HumanEval+86.6%—
MBPP+73.3%—

Agentic & Tool Use Not comparable

GPT-4 Turbo: —, Qwen2.5 14B Instruct: —

BenchmarkGPT-4 TurboQwen2.5 14B Instruct
METR Time Horizons36.7%—

Reasoning Not comparable

GPT-4 Turbo: 15.3 (#317), Qwen2.5 14B Instruct: —

BenchmarkGPT-4 TurboQwen2.5 14B Instruct
SimpleBench25.1%—
Chess Puzzles6%—
LMArena Hard Prompts1251—
DTBench61.6%—
LMCA9.8%—
Epoch Capabilities Index127.25—
ForecastBench59.4—

Math Not comparable

GPT-4 Turbo: 9.0 (#322), Qwen2.5 14B Instruct: —

BenchmarkGPT-4 TurboQwen2.5 14B Instruct
FrontierMath (Tiers 1-3)0.7%—
OTIS Mock AIME 2024-20256.7%—
LMArena Math1272—
MATH Level 546.7%—

Knowledge Not comparable

GPT-4 Turbo: 24.3 (#268), Qwen2.5 14B Instruct: —

BenchmarkGPT-4 TurboQwen2.5 14B Instruct
MMLU81.3%79.9%
GPQA Diamond46.6%—
Confabulations28.4%—
LMArena Expert1223—

Multimodal Not comparable

GPT-4 Turbo: 30.6 (#110), Qwen2.5 14B Instruct: —

BenchmarkGPT-4 TurboQwen2.5 14B Instruct
LMArena Vision1090—

Multilingual Not comparable

GPT-4 Turbo: 40.5 (#216), Qwen2.5 14B Instruct: —

BenchmarkGPT-4 TurboQwen2.5 14B Instruct
LMArena Non-English1245—
LMArena Chinese1242—
LMArena French1276—
LMArena German1259—
LMArena Japanese1194—
LMArena Korean1187—
LMArena Russian1259—
LMArena Spanish1260—

Instruction Following Not comparable

GPT-4 Turbo: 65.8 (#216), Qwen2.5 14B Instruct: —

BenchmarkGPT-4 TurboQwen2.5 14B Instruct
LMArena Instruction Following1249—

Long Context Not comparable

GPT-4 Turbo: 38.0 (#206), Qwen2.5 14B Instruct: —

BenchmarkGPT-4 TurboQwen2.5 14B Instruct
LMArena Longer Query1254—

Writing & Preference Not comparable

GPT-4 Turbo: 47.7 (#206), Qwen2.5 14B Instruct: —

BenchmarkGPT-4 TurboQwen2.5 14B Instruct
LMArena Text1272—
LMArena Creative Writing1269—
LMArena Multi-Turn1267—

Frequently asked questions

Is GPT-4 Turbo better than Qwen2.5 14B Instruct?

GPT-4 Turbo has enough public results to be ranked (#292); Qwen2.5 14B Instruct does not yet, so treat this comparison as directional.

Which is cheaper, GPT-4 Turbo or Qwen2.5 14B Instruct?

Qwen2.5 14B Instruct is cheaper. It lists at $0.35 per million input tokens and $1.40 per million output tokens; GPT-4 Turbo lists at $10 and $30.

Is GPT-4 Turbo or Qwen2.5 14B Instruct better for coding?

Qwen2.5 14B Instruct scores higher on coding benchmarks: 37.7 versus 33.8 in the Noometry coding category.

Which has the bigger context window?

Qwen2.5 14B Instruct does, with 131K tokens against 128K.

How many benchmarks do GPT-4 Turbo and Qwen2.5 14B Instruct share?

3 benchmarks have published results for both models. GPT-4 Turbo has 36 scored results on Noometry and Qwen2.5 14B Instruct has 3.

Related comparisons

Go deeper