Noometry

Model comparison

DeepSeek Coder 6.7B vs GPT-4

GPT-4 has enough public results to be ranked (#308); DeepSeek Coder 6.7B does not yet, so treat this comparison as directional.

Last verified . 7 shared benchmarks.

DeepSeek Coder 6.7B DeepSeek

37.1

Unranked Sparse

GPT-4 OpenAI

28.9

Rank #308 Confirmed

Summary

  • They share 7 benchmarks with published results for both. DeepSeek Coder 6.7B scores higher in 1 category and GPT-4 in 0 categories; one gap is clear of the uncertainty.
  • The widest gap is in coding, where DeepSeek Coder 6.7B leads 35.2 to 31.2.
  • The biggest single-benchmark swing is BigCodeBench Complete: 43.8% for DeepSeek Coder 6.7B and 57.2% for GPT-4.
  • DeepSeek Coder 6.7B has downloadable open weights; the other is API-only.

Side by side

DeepSeek Coder 6.7BGPT-4
ProviderDeepSeekOpenAI
Noometry Index37.128.9
Released2023-11-022023-03-14
WeightsOpenProprietary
Context window—8K
Max output—8K
Input $ / M tokens—$30
Output $ / M tokens—$60
Results tracked938

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding DeepSeek Coder 6.7B leads

DeepSeek Coder 6.7B: 35.2 (#216), GPT-4: 31.2 (#277)

BenchmarkDeepSeek Coder 6.7BGPT-4
BigCodeBench Instruct35.5%46%
BigCodeBench Complete43.8%57.2%
HumanEval+71.3%79.3%
WeirdML—12.4%
LMArena Coding—1254
MBPP+65.6%—

Agentic & Tool Use Not comparable

DeepSeek Coder 6.7B: —, GPT-4: —

BenchmarkDeepSeek Coder 6.7BGPT-4
METR Time Horizons—36.1%

Reasoning Not comparable

DeepSeek Coder 6.7B: —, GPT-4: 17.3 (#285)

BenchmarkDeepSeek Coder 6.7BGPT-4
Epoch Capabilities Index88.88125.88
WinoGrande57.6%87.5%
Chess Puzzles—4%
LMArena Hard Prompts—1241
Mystery Game Puzzles—12%
DTBench—62.7%
LMCA—17.1%
BIG-Bench Hard—75.1%
ForecastBench—57.8
HellaSwag—95.3%

Math Not comparable

DeepSeek Coder 6.7B: —, GPT-4: 10.1 (#304)

BenchmarkDeepSeek Coder 6.7BGPT-4
GSM8K21.3%92%
OTIS Mock AIME 2024-2025—1.1%
LMArena Math—1269
MATH Level 5—23%

Knowledge Not comparable

DeepSeek Coder 6.7B: —, GPT-4: 18.4 (#277)

BenchmarkDeepSeek Coder 6.7BGPT-4
MMLU36.4%86.4%
GPQA Diamond—35.7%
LMArena Expert—1211
ARC (AI2) Challenge36.4%—
TriviaQA—84.8%

Multilingual Not comparable

DeepSeek Coder 6.7B: —, GPT-4: 40.8 (#210)

BenchmarkDeepSeek Coder 6.7BGPT-4
LMArena Non-English—1246
LMArena Chinese—1242
LMArena French—1282
LMArena German—1251
LMArena Japanese—1209
LMArena Korean—1184
LMArena Russian—1251
LMArena Spanish—1261

Instruction Following Not comparable

DeepSeek Coder 6.7B: —, GPT-4: 65.5 (#217)

BenchmarkDeepSeek Coder 6.7BGPT-4
LMArena Instruction Following—1241

Long Context Not comparable

DeepSeek Coder 6.7B: —, GPT-4: 37.7 (#207)

BenchmarkDeepSeek Coder 6.7BGPT-4
LMArena Longer Query—1244

Writing & Preference Not comparable

DeepSeek Coder 6.7B: —, GPT-4: 35.0 (#263)

BenchmarkDeepSeek Coder 6.7BGPT-4
LMArena Text—1263
LMArena Creative Writing—1244
EQ-Bench Creative Writing—752
LMArena Multi-Turn—1257

Frequently asked questions

Is DeepSeek Coder 6.7B better than GPT-4?

GPT-4 has enough public results to be ranked (#308); DeepSeek Coder 6.7B does not yet, so treat this comparison as directional.

Is DeepSeek Coder 6.7B or GPT-4 better for coding?

DeepSeek Coder 6.7B scores higher on coding benchmarks: 35.2 versus 31.2 in the Noometry coding category.

How many benchmarks do DeepSeek Coder 6.7B and GPT-4 share?

7 benchmarks have published results for both models. DeepSeek Coder 6.7B has 9 scored results on Noometry and GPT-4 has 38.

Related comparisons

Go deeper