Noometry

Model comparison

DeepSeek LLM 67B vs ERNIE 5.1

ERNIE 5.1 is the stronger model overall, scoring 43.8 to 24.9 on the Noometry Index.

Last verified . 10 shared benchmarks.

DeepSeek LLM 67B DeepSeek

24.9

Rank #347 Confirmed

ERNIE 5.1 Baidu

43.8

Rank #83 Confirmed

Summary

  • They share 10 benchmarks with published results for both. DeepSeek LLM 67B scores higher in 0 categories and ERNIE 5.1 in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where ERNIE 5.1 leads 41.9 to 7.0.
  • DeepSeek LLM 67B has downloadable open weights; the other is API-only.

Side by side

DeepSeek LLM 67BERNIE 5.1
ProviderDeepSeekBaidu
Noometry Index24.943.8
Released2023-11-29—
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked1519

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding ERNIE 5.1 leads

DeepSeek LLM 67B: 31.9 (#278), ERNIE 5.1: 44.0 (#76)

BenchmarkDeepSeek LLM 67BERNIE 5.1
LMArena Coding10961488

Agentic & Tool Use Not comparable

DeepSeek LLM 67B: —, ERNIE 5.1: —

BenchmarkDeepSeek LLM 67BERNIE 5.1
LMArena Search—1227

Reasoning ERNIE 5.1 leads

DeepSeek LLM 67B: 16.5 (#304), ERNIE 5.1: 21.9 (#211)

BenchmarkDeepSeek LLM 67BERNIE 5.1
LMArena Hard Prompts10701481
NYT Connections (extended)—23.4%
Chess Puzzles0%—
Epoch Capabilities Index110.5—

Math ERNIE 5.1 leads

DeepSeek LLM 67B: 8.7 (#324), ERNIE 5.1: 40.3 (#92)

BenchmarkDeepSeek LLM 67BERNIE 5.1
LMArena Math11081481
OTIS Mock AIME 2024-20250.8%—
MATH Level 56.4%—

Knowledge ERNIE 5.1 leads

DeepSeek LLM 67B: 7.0 (#313), ERNIE 5.1: 41.9 (#102)

BenchmarkDeepSeek LLM 67BERNIE 5.1
GPQA Diamond24.6%—
LMArena Expert—1492

Multilingual ERNIE 5.1 leads

DeepSeek LLM 67B: 29.4 (#267), ERNIE 5.1: 55.5 (#29)

BenchmarkDeepSeek LLM 67BERNIE 5.1
LMArena Non-English10731454
LMArena Chinese11321508
LMArena French—1488
LMArena German—1470
LMArena Japanese—1422
LMArena Korean—1427
LMArena Russian—1459
LMArena Spanish—1473

Instruction Following ERNIE 5.1 leads

DeepSeek LLM 67B: 55.4 (#277), ERNIE 5.1: 76.7 (#37)

BenchmarkDeepSeek LLM 67BERNIE 5.1
LMArena Instruction Following10791460

Long Context ERNIE 5.1 leads

DeepSeek LLM 67B: 33.1 (#265), ERNIE 5.1: 44.7 (#59)

BenchmarkDeepSeek LLM 67BERNIE 5.1
LMArena Longer Query10921462

Writing & Preference ERNIE 5.1 leads

DeepSeek LLM 67B: 31.6 (#282), ERNIE 5.1: 65.1 (#52)

BenchmarkDeepSeek LLM 67BERNIE 5.1
LMArena Text11051468
LMArena Creative Writing10671441
LMArena Multi-Turn10821471

Frequently asked questions

Is DeepSeek LLM 67B better than ERNIE 5.1?

ERNIE 5.1 is the stronger model overall, scoring 43.8 to 24.9 on the Noometry Index.

Is DeepSeek LLM 67B or ERNIE 5.1 better for coding?

ERNIE 5.1 scores higher on coding benchmarks: 44.0 versus 31.9 in the Noometry coding category.

How many benchmarks do DeepSeek LLM 67B and ERNIE 5.1 share?

10 benchmarks have published results for both models. DeepSeek LLM 67B has 15 scored results on Noometry and ERNIE 5.1 has 19.

Related comparisons

Go deeper