Noometry

Model comparison

Deepseek Coder v2 vs GPT-4 Turbo

Deepseek Coder v2 is the stronger model overall, scoring 35.3 to 30.3 on the Noometry Index.

Last verified . 21 shared benchmarks.

Deepseek Coder v2 DeepSeek

35.3

Rank #217 Confirmed

GPT-4 Turbo OpenAI

30.3

Rank #281 Confirmed

Summary

  • They share 21 benchmarks with published results for both. Deepseek Coder v2 scores higher in 4 categories and GPT-4 Turbo in 4 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Deepseek Coder v2 leads 32.3 to 8.6.
  • Deepseek Coder v2 has downloadable open weights; the other is API-only.

Side by side

Deepseek Coder v2GPT-4 Turbo
ProviderDeepSeekOpenAI
Noometry Index35.330.3
Released2024-06-172023-11-06
WeightsOpenProprietary
Context window—128K
Max output—4K
Input $ / M tokens—$10
Output $ / M tokens—$30
Results tracked2436

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Deepseek Coder v2 leads

Deepseek Coder v2: 37.5 (#178), GPT-4 Turbo: 33.4 (#247)

BenchmarkDeepseek Coder v2GPT-4 Turbo
BigCodeBench Instruct48.2%48.2%
LMArena Coding12511268
BigCodeBench Complete59.7%58.2%
HumanEval+82.3%86.6%
MBPP+75.1%73.3%
WeirdML—18%

Agentic & Tool Use Not comparable

Deepseek Coder v2: —, GPT-4 Turbo: —

BenchmarkDeepseek Coder v2GPT-4 Turbo
METR Time Horizons—36.7%

Reasoning Deepseek Coder v2 leads

Deepseek Coder v2: 22.6 (#175), GPT-4 Turbo: 14.9 (#311)

BenchmarkDeepseek Coder v2GPT-4 Turbo
LMArena Hard Prompts12071251
SimpleBench—25.1%
Chess Puzzles—6%
DTBench—61.6%
LMCA—9.8%
Epoch Capabilities Index—127.25
ForecastBench—59.4
WinoGrande83.7%—

Math Deepseek Coder v2 leads

Deepseek Coder v2: 32.3 (#188), GPT-4 Turbo: 8.6 (#316)

BenchmarkDeepseek Coder v2GPT-4 Turbo
LMArena Math12411272
FrontierMath (Tiers 1-3)—0.7%
OTIS Mock AIME 2024-2025—6.7%
MATH Level 5—46.7%
GSM8K94.5%—

Knowledge Deepseek Coder v2 leads

Deepseek Coder v2: 31.7 (#209), GPT-4 Turbo: 24.3 (#263)

BenchmarkDeepseek Coder v2GPT-4 Turbo
LMArena Expert11811223
GPQA Diamond—46.6%
Confabulations—28.4%
ARC (AI2) Challenge64.3%—
MMLU—81.3%

Multimodal Not comparable

Deepseek Coder v2: —, GPT-4 Turbo: 30.0 (#104)

BenchmarkDeepseek Coder v2GPT-4 Turbo
LMArena Vision—1090

Multilingual GPT-4 Turbo leads

Deepseek Coder v2: 36.4 (#235), GPT-4 Turbo: 40.7 (#211)

BenchmarkDeepseek Coder v2GPT-4 Turbo
LMArena Non-English11821245
LMArena Chinese12011242
LMArena French11841275
LMArena German11641259
LMArena Japanese11261194
LMArena Korean11041187
LMArena Russian11881259
LMArena Spanish11531260

Instruction Following GPT-4 Turbo leads

Deepseek Coder v2: 61.9 (#237), GPT-4 Turbo: 66.0 (#211)

BenchmarkDeepseek Coder v2GPT-4 Turbo
LMArena Instruction Following11801250

Long Context GPT-4 Turbo leads

Deepseek Coder v2: 37.0 (#219), GPT-4 Turbo: 38.0 (#200)

BenchmarkDeepseek Coder v2GPT-4 Turbo
LMArena Longer Query12191254

Writing & Preference GPT-4 Turbo leads

Deepseek Coder v2: 38.4 (#249), GPT-4 Turbo: 47.9 (#201)

BenchmarkDeepseek Coder v2GPT-4 Turbo
LMArena Text11911272
LMArena Creative Writing11201269
LMArena Multi-Turn11761267

Frequently asked questions

Is Deepseek Coder v2 better than GPT-4 Turbo?

Deepseek Coder v2 is the stronger model overall, scoring 35.3 to 30.3 on the Noometry Index.

Is Deepseek Coder v2 or GPT-4 Turbo better for coding?

Deepseek Coder v2 scores higher on coding benchmarks: 37.5 versus 33.4 in the Noometry coding category.

How many benchmarks do Deepseek Coder v2 and GPT-4 Turbo share?

21 benchmarks have published results for both models. Deepseek Coder v2 has 24 scored results on Noometry and GPT-4 Turbo has 36.

Related comparisons

Go deeper