Noometry

Model comparison

Deepseek Coder v2 vs GPT-3.5-turbo

Deepseek Coder v2 is the stronger model overall, scoring 35.9 to 23.2 on the Noometry Index.

Last verified . 24 shared benchmarks.

Deepseek Coder v2 DeepSeek

35.9

Rank #220 Confirmed

GPT-3.5-turbo OpenAI

23.2

Rank #350 Confirmed

Summary

  • They share 24 benchmarks with published results for both. Deepseek Coder v2 scores higher in 8 categories and GPT-3.5-turbo in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Deepseek Coder v2 leads 34.9 to 6.3.
  • The biggest single-benchmark swing is BigCodeBench Complete: 59.7% for Deepseek Coder v2 and 50.6% for GPT-3.5-turbo.
  • Deepseek Coder v2 has downloadable open weights; the other is API-only.

Side by side

Deepseek Coder v2GPT-3.5-turbo
ProviderDeepSeekOpenAI
Noometry Index35.923.2
Released2024-06-172023-03-01
WeightsOpenProprietary
Context window—16K
Max output—4K
Input $ / M tokens—$0.50
Output $ / M tokens—$1.50
Results tracked2444

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Deepseek Coder v2 leads

Deepseek Coder v2: 38.1 (#183), GPT-3.5-turbo: 23.9 (#331)

BenchmarkDeepseek Coder v2GPT-3.5-turbo
BigCodeBench Instruct48.2%39.1%
LMArena Coding12511136
BigCodeBench Complete59.7%50.6%
HumanEval+82.3%70.7%
MBPP+75.1%69.7%
WeirdML—3.5%

Agentic & Tool Use Not comparable

Deepseek Coder v2: —, GPT-3.5-turbo: —

BenchmarkDeepseek Coder v2GPT-3.5-turbo
METR Time Horizons—21.5%

Reasoning Deepseek Coder v2 leads

Deepseek Coder v2: 23.6 (#176), GPT-3.5-turbo: 13.8 (#332)

BenchmarkDeepseek Coder v2GPT-3.5-turbo
LMArena Hard Prompts12071108
WinoGrande83.7%81.6%
Chess Puzzles—0%
Mystery Game Puzzles—3%
DTBench—48.5%
LMCA—9.7%
Adversarial NLI—58.1%
BIG-Bench Hard—61.6%
CommonsenseQA 2.0—57%
Epoch Capabilities Index—118.55
ForecastBench—50.4

Math Deepseek Coder v2 leads

Deepseek Coder v2: 34.9 (#190), GPT-3.5-turbo: 6.3 (#327)

BenchmarkDeepseek Coder v2GPT-3.5-turbo
LMArena Math12411142
GSM8K94.5%57.8%
FrontierMath (Tiers 1-3)—0%
OTIS Mock AIME 2024-2025—2.2%
MATH Level 5—15.9%

Knowledge Deepseek Coder v2 leads

Deepseek Coder v2: 32.3 (#212), GPT-3.5-turbo: 10.0 (#303)

BenchmarkDeepseek Coder v2GPT-3.5-turbo
LMArena Expert11811070
ARC (AI2) Challenge64.3%87.4%
GPQA Diamond—28%
BoolQ—87%
MMLU—71.4%
OpenBookQA—86%
TriviaQA—85.8%

Multilingual Deepseek Coder v2 leads

Deepseek Coder v2: 36.3 (#240), GPT-3.5-turbo: 31.5 (#258)

BenchmarkDeepseek Coder v2GPT-3.5-turbo
LMArena Non-English11821108
LMArena Chinese12011075
LMArena French11851118
LMArena German11641090
LMArena Japanese11261043
LMArena Korean11041019
LMArena Russian11881123
LMArena Spanish11531121

Instruction Following Deepseek Coder v2 leads

Deepseek Coder v2: 61.7 (#242), GPT-3.5-turbo: 57.9 (#262)

BenchmarkDeepseek Coder v2GPT-3.5-turbo
LMArena Instruction Following11801119

Long Context Deepseek Coder v2 leads

Deepseek Coder v2: 37.0 (#224), GPT-3.5-turbo: 34.0 (#254)

BenchmarkDeepseek Coder v2GPT-3.5-turbo
LMArena Longer Query12191121

Writing & Preference Deepseek Coder v2 leads

Deepseek Coder v2: 38.2 (#253), GPT-3.5-turbo: 25.3 (#305)

BenchmarkDeepseek Coder v2GPT-3.5-turbo
LMArena Text11911125
LMArena Creative Writing11201092
LMArena Multi-Turn11771117
EQ-Bench Creative Writing—451

Frequently asked questions

Is Deepseek Coder v2 better than GPT-3.5-turbo?

Deepseek Coder v2 is the stronger model overall, scoring 35.9 to 23.2 on the Noometry Index.

Is Deepseek Coder v2 or GPT-3.5-turbo better for coding?

Deepseek Coder v2 scores higher on coding benchmarks: 38.1 versus 23.9 in the Noometry coding category.

How many benchmarks do Deepseek Coder v2 and GPT-3.5-turbo share?

24 benchmarks have published results for both models. Deepseek Coder v2 has 24 scored results on Noometry and GPT-3.5-turbo has 44.

Related comparisons

Go deeper