Noometry

Model comparison

DeepSeek LLM 67B vs o3-mini

o3-mini is the stronger model overall, scoring 36.7 to 24.9 on the Noometry Index.

Last verified . 15 shared benchmarks.

DeepSeek LLM 67B DeepSeek

24.9

Rank #347 Confirmed

o3-mini OpenAI

36.7

Rank #212 Confirmed

Summary

  • They share 15 benchmarks with published results for both. DeepSeek LLM 67B scores higher in 1 category and o3-mini in 7 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where o3-mini leads 38.3 to 7.0.
  • The biggest single-benchmark swing is MATH Level 5: 6.4% for DeepSeek LLM 67B and 96.5% for o3-mini.
  • DeepSeek LLM 67B has downloadable open weights; the other is API-only.

Side by side

DeepSeek LLM 67Bo3-mini
ProviderDeepSeekOpenAI
Noometry Index24.936.7
Released2023-11-292024-12-20
WeightsOpenProprietary
Context window—200K
Max output—100K
Input $ / M tokens—$1.10
Output $ / M tokens—$4.40
Results tracked1551

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3-mini leads

DeepSeek LLM 67B: 31.9 (#278), o3-mini: 40.8 (#132)

BenchmarkDeepSeek LLM 67Bo3-mini
LMArena Coding10961378
Aider Polyglot—60.4%
SciCode—39.8%
GSO—1.3%
WeirdML—43.7%
LiveBench Coding—82.7%
CadEval—54%

Agentic & Tool Use Not comparable

DeepSeek LLM 67B: —, o3-mini: 29.6 (#84)

BenchmarkDeepSeek LLM 67Bo3-mini
Cybench—22.5%

Reasoning Too close to call

DeepSeek LLM 67B: 16.5 (#304), o3-mini: 16.3 (#305)

BenchmarkDeepSeek LLM 67Bo3-mini
Chess Puzzles0%17%
LMArena Hard Prompts10701366
Epoch Capabilities Index110.5140.34
ARC-AGI-2—3%
SimpleBench—22.8%
ARC-AGI-1—34.5%
CritPt—0.3%
LiveBench Reasoning—89.6%
Mystery Game Puzzles—7%
DTBench—68.8%
LiveBench Data Analysis—70.6%
LMCA—19%
ForecastBench—59.6
LiveBench—75.9%

Math o3-mini leads

DeepSeek LLM 67B: 8.7 (#324), o3-mini: 28.1 (#244)

BenchmarkDeepSeek LLM 67Bo3-mini
OTIS Mock AIME 2024-20250.8%76.9%
LMArena Math11081396
MATH Level 56.4%96.5%
FrontierMath (Tiers 1-3)—18.6%
FrontierMath Tier 4—0%
LiveBench Math—77.3%
FrontierMath (Feb 2025 set)—12.4%
FrontierMath Tier 4 (v1)—4.2%

Knowledge o3-mini leads

DeepSeek LLM 67B: 7.0 (#313), o3-mini: 38.3 (#146)

BenchmarkDeepSeek LLM 67Bo3-mini
GPQA Diamond24.6%77%
SimpleQA Verified—15.3%
Confabulations—17.9%
LMArena Expert—1364

Multilingual o3-mini leads

DeepSeek LLM 67B: 29.4 (#267), o3-mini: 45.7 (#164)

BenchmarkDeepSeek LLM 67Bo3-mini
LMArena Non-English10731319
LMArena Chinese11321379
LMArena French—1334
LMArena German—1303
LMArena Japanese—1286
LMArena Korean—1314
LMArena Russian—1304
LMArena Spanish—1321

Instruction Following o3-mini leads

DeepSeek LLM 67B: 55.4 (#277), o3-mini: 75.1 (#72)

BenchmarkDeepSeek LLM 67Bo3-mini
LMArena Instruction Following10791337
LiveBench Instruction Following—84.4%

Long Context Too close to call

DeepSeek LLM 67B: 33.1 (#265), o3-mini: 33.8 (#256)

BenchmarkDeepSeek LLM 67Bo3-mini
LMArena Longer Query10921343
Fiction.LiveBench—50%

Writing & Preference o3-mini leads

DeepSeek LLM 67B: 31.6 (#282), o3-mini: 50.3 (#182)

BenchmarkDeepSeek LLM 67Bo3-mini
LMArena Text11051337
LMArena Creative Writing10671286
LMArena Multi-Turn10821320
Short-Story Creative Writing—61.7%
LiveBench Language—50.7%

Frequently asked questions

Is DeepSeek LLM 67B better than o3-mini?

o3-mini is the stronger model overall, scoring 36.7 to 24.9 on the Noometry Index.

Is DeepSeek LLM 67B or o3-mini better for coding?

o3-mini scores higher on coding benchmarks: 40.8 versus 31.9 in the Noometry coding category.

How many benchmarks do DeepSeek LLM 67B and o3-mini share?

15 benchmarks have published results for both models. DeepSeek LLM 67B has 15 scored results on Noometry and o3-mini has 51.

Related comparisons

Go deeper