Noometry

Model comparison

DeepSeek Coder 6.7B vs Llama 3.1-70B

Llama 3.1-70B has enough public results to be ranked (#308); DeepSeek Coder 6.7B does not yet, so treat this comparison as directional.

Last verified . 4 shared benchmarks.

DeepSeek Coder 6.7B DeepSeek

37.6

Unranked Sparse

Llama 3.1-70B Meta

29.6

Rank #308 Confirmed

Summary

  • They share 4 benchmarks with published results for both. DeepSeek Coder 6.7B scores higher in 1 category and Llama 3.1-70B in 0 categories; one gap is clear of the uncertainty.
  • The widest gap is in coding, where DeepSeek Coder 6.7B leads 35.8 to 30.3.
  • The biggest single-benchmark swing is BigCodeBench Complete: 43.8% for DeepSeek Coder 6.7B and 54.8% for Llama 3.1-70B.

Side by side

DeepSeek Coder 6.7BLlama 3.1-70B
ProviderDeepSeekMeta
Noometry Index37.629.6
Released2023-11-022024-07-23
WeightsOpenOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.40
Output $ / M tokens—$0.40
Results tracked935

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding DeepSeek Coder 6.7B leads

DeepSeek Coder 6.7B: 35.8 (#219), Llama 3.1-70B: 30.3 (#296)

BenchmarkDeepSeek Coder 6.7BLlama 3.1-70B
BigCodeBench Instruct35.5%46.1%
BigCodeBench Complete43.8%54.8%
WeirdML—9%
LMArena Coding—1260
HumanEval+71.3%—
MBPP+65.6%—

Agentic & Tool Use Not comparable

DeepSeek Coder 6.7B: —, Llama 3.1-70B: 25.1 (#112)

BenchmarkDeepSeek Coder 6.7BLlama 3.1-70B
TheAgentCompany—6.9%
BALROG—27.9%

Reasoning Not comparable

DeepSeek Coder 6.7B: —, Llama 3.1-70B: 21.6 (#220)

BenchmarkDeepSeek Coder 6.7BLlama 3.1-70B
Epoch Capabilities Index89.62125.92
LMArena Hard Prompts—1241
DTBench—60%
LMCA—14.8%
WinoGrande57.6%—

Math Not comparable

DeepSeek Coder 6.7B: —, Llama 3.1-70B: 13.5 (#304)

BenchmarkDeepSeek Coder 6.7BLlama 3.1-70B
OTIS Mock AIME 2024-2025—3.6%
Omni-MATH—21%
LMArena Math—1252
MATH Level 5—36.7%
GSM8K21.3%—

Knowledge Not comparable

DeepSeek Coder 6.7B: —, Llama 3.1-70B: 24.2 (#269)

BenchmarkDeepSeek Coder 6.7BLlama 3.1-70B
MMLU36.4%80.1%
GPQA Diamond—44.2%
MMLU-Pro—65.3%
GPQA (HELM)—42.6%
LMArena Expert—1209
ARC (AI2) Challenge36.4%—

Multilingual Not comparable

DeepSeek Coder 6.7B: —, Llama 3.1-70B: 38.8 (#225)

BenchmarkDeepSeek Coder 6.7BLlama 3.1-70B
LMArena Non-English—1219
LMArena Chinese—1215
LMArena French—1261
LMArena German—1222
LMArena Japanese—1132
LMArena Korean—1140
LMArena Russian—1234
LMArena Spanish—1253

Instruction Following Not comparable

DeepSeek Coder 6.7B: —, Llama 3.1-70B: 65.3 (#223)

BenchmarkDeepSeek Coder 6.7BLlama 3.1-70B
IFEval—82.1%
LMArena Instruction Following—1231

Long Context Not comparable

DeepSeek Coder 6.7B: —, Llama 3.1-70B: 37.6 (#214)

BenchmarkDeepSeek Coder 6.7BLlama 3.1-70B
LMArena Longer Query—1241

Writing & Preference Not comparable

DeepSeek Coder 6.7B: —, Llama 3.1-70B: 35.4 (#267)

BenchmarkDeepSeek Coder 6.7BLlama 3.1-70B
LMArena Text—1261
LMArena Creative Writing—1232
EQ-Bench Creative Writing—784
WildBench—75.8%
LMArena Multi-Turn—1256

Frequently asked questions

Is DeepSeek Coder 6.7B better than Llama 3.1-70B?

Llama 3.1-70B has enough public results to be ranked (#308); DeepSeek Coder 6.7B does not yet, so treat this comparison as directional.

Is DeepSeek Coder 6.7B or Llama 3.1-70B better for coding?

DeepSeek Coder 6.7B scores higher on coding benchmarks: 35.8 versus 30.3 in the Noometry coding category.

How many benchmarks do DeepSeek Coder 6.7B and Llama 3.1-70B share?

4 benchmarks have published results for both models. DeepSeek Coder 6.7B has 9 scored results on Noometry and Llama 3.1-70B has 35.

Related comparisons

Go deeper