Noometry

Model comparison

DeepSeek LLM 67B vs GPT-3.5-turbo

DeepSeek LLM 67B is the stronger model overall, scoring 24.9 to 23.2 on the Noometry Index.

Last verified . 15 shared benchmarks.

DeepSeek LLM 67B DeepSeek

24.9

Rank #347 Confirmed

GPT-3.5-turbo OpenAI

23.2

Rank #350 Confirmed

Summary

  • They share 15 benchmarks with published results for both. DeepSeek LLM 67B scores higher in 4 categories and GPT-3.5-turbo in 4 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in coding, where DeepSeek LLM 67B leads 31.9 to 23.9.
  • The biggest single-benchmark swing is MATH Level 5: 6.4% for DeepSeek LLM 67B and 15.9% for GPT-3.5-turbo.
  • DeepSeek LLM 67B has downloadable open weights; the other is API-only.

Side by side

DeepSeek LLM 67BGPT-3.5-turbo
ProviderDeepSeekOpenAI
Noometry Index24.923.2
Released2023-11-292023-03-01
WeightsOpenProprietary
Context window—16K
Max output—4K
Input $ / M tokens—$0.50
Output $ / M tokens—$1.50
Results tracked1544

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding DeepSeek LLM 67B leads

DeepSeek LLM 67B: 31.9 (#278), GPT-3.5-turbo: 23.9 (#331)

BenchmarkDeepSeek LLM 67BGPT-3.5-turbo
LMArena Coding10961136
WeirdML—3.5%
BigCodeBench Instruct—39.1%
BigCodeBench Complete—50.6%
HumanEval+—70.7%
MBPP+—69.7%

Agentic & Tool Use Not comparable

DeepSeek LLM 67B: —, GPT-3.5-turbo: —

BenchmarkDeepSeek LLM 67BGPT-3.5-turbo
METR Time Horizons—21.5%

Reasoning DeepSeek LLM 67B leads

DeepSeek LLM 67B: 16.5 (#304), GPT-3.5-turbo: 13.8 (#332)

BenchmarkDeepSeek LLM 67BGPT-3.5-turbo
Chess Puzzles0%0%
LMArena Hard Prompts10701108
Epoch Capabilities Index110.5118.55
Mystery Game Puzzles—3%
DTBench—48.5%
LMCA—9.7%
Adversarial NLI—58.1%
BIG-Bench Hard—61.6%
CommonsenseQA 2.0—57%
ForecastBench—50.4
WinoGrande—81.6%

Math DeepSeek LLM 67B leads

DeepSeek LLM 67B: 8.7 (#324), GPT-3.5-turbo: 6.3 (#327)

BenchmarkDeepSeek LLM 67BGPT-3.5-turbo
OTIS Mock AIME 2024-20250.8%2.2%
LMArena Math11081142
MATH Level 56.4%15.9%
FrontierMath (Tiers 1-3)—0%
GSM8K—57.8%

Knowledge GPT-3.5-turbo leads

DeepSeek LLM 67B: 7.0 (#313), GPT-3.5-turbo: 10.0 (#303)

BenchmarkDeepSeek LLM 67BGPT-3.5-turbo
GPQA Diamond24.6%28%
LMArena Expert—1070
ARC (AI2) Challenge—87.4%
BoolQ—87%
MMLU—71.4%
OpenBookQA—86%
TriviaQA—85.8%

Multilingual GPT-3.5-turbo leads

DeepSeek LLM 67B: 29.4 (#267), GPT-3.5-turbo: 31.5 (#258)

BenchmarkDeepSeek LLM 67BGPT-3.5-turbo
LMArena Non-English10731108
LMArena Chinese11321075
LMArena French—1118
LMArena German—1090
LMArena Japanese—1043
LMArena Korean—1019
LMArena Russian—1123
LMArena Spanish—1121

Instruction Following GPT-3.5-turbo leads

DeepSeek LLM 67B: 55.4 (#277), GPT-3.5-turbo: 57.9 (#262)

BenchmarkDeepSeek LLM 67BGPT-3.5-turbo
LMArena Instruction Following10791119

Long Context Too close to call

DeepSeek LLM 67B: 33.1 (#265), GPT-3.5-turbo: 34.0 (#254)

BenchmarkDeepSeek LLM 67BGPT-3.5-turbo
LMArena Longer Query10921121

Writing & Preference DeepSeek LLM 67B leads

DeepSeek LLM 67B: 31.6 (#282), GPT-3.5-turbo: 25.3 (#305)

BenchmarkDeepSeek LLM 67BGPT-3.5-turbo
LMArena Text11051125
LMArena Creative Writing10671092
LMArena Multi-Turn10821117
EQ-Bench Creative Writing—451

Frequently asked questions

Is DeepSeek LLM 67B better than GPT-3.5-turbo?

DeepSeek LLM 67B is the stronger model overall, scoring 24.9 to 23.2 on the Noometry Index.

Is DeepSeek LLM 67B or GPT-3.5-turbo better for coding?

DeepSeek LLM 67B scores higher on coding benchmarks: 31.9 versus 23.9 in the Noometry coding category.

How many benchmarks do DeepSeek LLM 67B and GPT-3.5-turbo share?

15 benchmarks have published results for both models. DeepSeek LLM 67B has 15 scored results on Noometry and GPT-3.5-turbo has 44.

Related comparisons

Go deeper