Noometry

Model comparison

DeepSeek LLM 67B vs GPT-4 Turbo

GPT-4 Turbo is the stronger model overall, scoring 30.3 to 24.8 on the Noometry Index.

Last verified . 15 shared benchmarks.

DeepSeek LLM 67B DeepSeek

24.8

Rank #341 Confirmed

GPT-4 Turbo OpenAI

30.3

Rank #281 Confirmed

Summary

  • They share 15 benchmarks with published results for both. DeepSeek LLM 67B scores higher in 1 category and GPT-4 Turbo in 7 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where GPT-4 Turbo leads 24.3 to 7.1.
  • The biggest single-benchmark swing is MATH Level 5: 6.4% for DeepSeek LLM 67B and 46.7% for GPT-4 Turbo.
  • DeepSeek LLM 67B has downloadable open weights; the other is API-only.

Side by side

DeepSeek LLM 67BGPT-4 Turbo
ProviderDeepSeekOpenAI
Noometry Index24.830.3
Released2023-11-292023-11-06
WeightsOpenProprietary
Context window—128K
Max output—4K
Input $ / M tokens—$10
Output $ / M tokens—$30
Results tracked1536

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4 Turbo leads

DeepSeek LLM 67B: 31.3 (#273), GPT-4 Turbo: 33.4 (#247)

BenchmarkDeepSeek LLM 67BGPT-4 Turbo
LMArena Coding10961268
WeirdML—18%
BigCodeBench Instruct—48.2%
BigCodeBench Complete—58.2%
HumanEval+—86.6%
MBPP+—73.3%

Agentic & Tool Use Not comparable

DeepSeek LLM 67B: —, GPT-4 Turbo: —

BenchmarkDeepSeek LLM 67BGPT-4 Turbo
METR Time Horizons—36.7%

Reasoning DeepSeek LLM 67B leads

DeepSeek LLM 67B: 15.9 (#300), GPT-4 Turbo: 14.9 (#311)

BenchmarkDeepSeek LLM 67BGPT-4 Turbo
Chess Puzzles0%6%
LMArena Hard Prompts10701251
Epoch Capabilities Index110.54127.25
SimpleBench—25.1%
DTBench—61.6%
LMCA—9.8%
ForecastBench—59.4

Math Too close to call

DeepSeek LLM 67B: 8.1 (#318), GPT-4 Turbo: 8.6 (#316)

BenchmarkDeepSeek LLM 67BGPT-4 Turbo
OTIS Mock AIME 2024-20250.8%6.7%
LMArena Math11081272
MATH Level 56.4%46.7%
FrontierMath (Tiers 1-3)—0.7%

Knowledge GPT-4 Turbo leads

DeepSeek LLM 67B: 7.1 (#308), GPT-4 Turbo: 24.3 (#263)

BenchmarkDeepSeek LLM 67BGPT-4 Turbo
GPQA Diamond24.6%46.6%
Confabulations—28.4%
LMArena Expert—1223
MMLU—81.3%

Multimodal Not comparable

DeepSeek LLM 67B: —, GPT-4 Turbo: 30.0 (#104)

BenchmarkDeepSeek LLM 67BGPT-4 Turbo
LMArena Vision—1090

Multilingual GPT-4 Turbo leads

DeepSeek LLM 67B: 29.5 (#262), GPT-4 Turbo: 40.7 (#211)

BenchmarkDeepSeek LLM 67BGPT-4 Turbo
LMArena Non-English10731245
LMArena Chinese11321242
LMArena French—1275
LMArena German—1259
LMArena Japanese—1194
LMArena Korean—1187
LMArena Russian—1259
LMArena Spanish—1260

Instruction Following GPT-4 Turbo leads

DeepSeek LLM 67B: 55.7 (#272), GPT-4 Turbo: 66.0 (#211)

BenchmarkDeepSeek LLM 67BGPT-4 Turbo
LMArena Instruction Following10791250

Long Context GPT-4 Turbo leads

DeepSeek LLM 67B: 33.1 (#260), GPT-4 Turbo: 38.0 (#200)

BenchmarkDeepSeek LLM 67BGPT-4 Turbo
LMArena Longer Query10921254

Writing & Preference GPT-4 Turbo leads

DeepSeek LLM 67B: 31.8 (#277), GPT-4 Turbo: 47.9 (#201)

BenchmarkDeepSeek LLM 67BGPT-4 Turbo
LMArena Text11051272
LMArena Creative Writing10671269
LMArena Multi-Turn10821267

Frequently asked questions

Is DeepSeek LLM 67B better than GPT-4 Turbo?

GPT-4 Turbo is the stronger model overall, scoring 30.3 to 24.8 on the Noometry Index.

Is DeepSeek LLM 67B or GPT-4 Turbo better for coding?

GPT-4 Turbo scores higher on coding benchmarks: 33.4 versus 31.3 in the Noometry coding category.

How many benchmarks do DeepSeek LLM 67B and GPT-4 Turbo share?

15 benchmarks have published results for both models. DeepSeek LLM 67B has 15 scored results on Noometry and GPT-4 Turbo has 36.

Related comparisons

Go deeper