Noometry

Model comparison

GPT-3.5-turbo vs Qwen1.5-32B

Qwen1.5-32B is the stronger model overall, scoring 30.5 to 23.2 on the Noometry Index.

Last verified . 21 shared benchmarks.

GPT-3.5-turbo OpenAI

23.2

Rank #350 Confirmed

Qwen1.5-32B Alibaba (Qwen)

30.5

Rank #293 Confirmed

Summary

  • They share 21 benchmarks with published results for both. GPT-3.5-turbo scores higher in 2 categories and Qwen1.5-32B in 6 categories; 5 gaps are clear of the uncertainty.
  • The widest gap is in math, where Qwen1.5-32B leads 33.0 to 6.3.
  • The biggest single-benchmark swing is BigCodeBench Complete: 50.6% for GPT-3.5-turbo and 42% for Qwen1.5-32B.
  • Qwen1.5-32B has downloadable open weights; the other is API-only.

Side by side

GPT-3.5-turboQwen1.5-32B
ProviderOpenAIAlibaba (Qwen)
Noometry Index23.230.5
Released2023-03-012024-02-04
WeightsProprietaryOpen
Context window16K—
Max output4K—
Input $ / M tokens$0.50—
Output $ / M tokens$1.50—
Results tracked4421

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen1.5-32B leads

GPT-3.5-turbo: 23.9 (#331), Qwen1.5-32B: 31.7 (#282)

BenchmarkGPT-3.5-turboQwen1.5-32B
BigCodeBench Instruct39.1%32.3%
LMArena Coding11361155
BigCodeBench Complete50.6%42%
WeirdML3.5%—
HumanEval+70.7%—
MBPP+69.7%—

Agentic & Tool Use Not comparable

GPT-3.5-turbo: —, Qwen1.5-32B: —

BenchmarkGPT-3.5-turboQwen1.5-32B
METR Time Horizons21.5%—

Reasoning Qwen1.5-32B leads

GPT-3.5-turbo: 13.8 (#332), Qwen1.5-32B: 21.8 (#212)

BenchmarkGPT-3.5-turboQwen1.5-32B
LMArena Hard Prompts11081130
Chess Puzzles0%—
Mystery Game Puzzles3%—
DTBench48.5%—
LMCA9.7%—
Adversarial NLI58.1%—
BIG-Bench Hard61.6%—
CommonsenseQA 2.057%—
Epoch Capabilities Index118.55—
ForecastBench50.4—
WinoGrande81.6%—

Math Qwen1.5-32B leads

GPT-3.5-turbo: 6.3 (#327), Qwen1.5-32B: 33.0 (#207)

BenchmarkGPT-3.5-turboQwen1.5-32B
LMArena Math11421155
FrontierMath (Tiers 1-3)0%—
OTIS Mock AIME 2024-20252.2%—
MATH Level 515.9%—
GSM8K57.8%—

Knowledge Qwen1.5-32B leads

GPT-3.5-turbo: 10.0 (#303), Qwen1.5-32B: 13.5 (#296)

BenchmarkGPT-3.5-turboQwen1.5-32B
GPQA Diamond28%30.7%
LMArena Expert10701126
MMLU71.4%74.4%
ARC (AI2) Challenge87.4%—
BoolQ87%—
OpenBookQA86%—
TriviaQA85.8%—

Multilingual Too close to call

GPT-3.5-turbo: 31.5 (#258), Qwen1.5-32B: 31.4 (#259)

BenchmarkGPT-3.5-turboQwen1.5-32B
LMArena Non-English11081106
LMArena Chinese10751177
LMArena French11181101
LMArena German10901058
LMArena Japanese10431027
LMArena Korean10191008
LMArena Russian11231073
LMArena Spanish11211089

Instruction Following Too close to call

GPT-3.5-turbo: 57.9 (#262), Qwen1.5-32B: 57.7 (#265)

BenchmarkGPT-3.5-turboQwen1.5-32B
LMArena Instruction Following11191116

Long Context Too close to call

GPT-3.5-turbo: 34.0 (#254), Qwen1.5-32B: 34.7 (#246)

BenchmarkGPT-3.5-turboQwen1.5-32B
LMArena Longer Query11211146

Writing & Preference Qwen1.5-32B leads

GPT-3.5-turbo: 25.3 (#305), Qwen1.5-32B: 34.2 (#271)

BenchmarkGPT-3.5-turboQwen1.5-32B
LMArena Text11251137
LMArena Creative Writing10921083
LMArena Multi-Turn11171140
EQ-Bench Creative Writing451—

Frequently asked questions

Is GPT-3.5-turbo better than Qwen1.5-32B?

Qwen1.5-32B is the stronger model overall, scoring 30.5 to 23.2 on the Noometry Index.

Is GPT-3.5-turbo or Qwen1.5-32B better for coding?

Qwen1.5-32B scores higher on coding benchmarks: 31.7 versus 23.9 in the Noometry coding category.

How many benchmarks do GPT-3.5-turbo and Qwen1.5-32B share?

21 benchmarks have published results for both models. GPT-3.5-turbo has 44 scored results on Noometry and Qwen1.5-32B has 21.

Related comparisons

Go deeper