Noometry

Model comparison

Deepseek Coder v2 vs Llama 3.2 3B

Deepseek Coder v2 is the stronger model overall, scoring 35.3 to 28.4 on the Noometry Index.

Last verified . 15 shared benchmarks.

Deepseek Coder v2 DeepSeek

35.3

Rank #217 Confirmed

Llama 3.2 3B Meta

28.4

Rank #318 Confirmed

Summary

  • They share 15 benchmarks with published results for both. Deepseek Coder v2 scores higher in 8 categories and Llama 3.2 3B in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Deepseek Coder v2 leads 38.4 to 24.9.
  • The biggest single-benchmark swing is BigCodeBench Complete: 59.7% for Deepseek Coder v2 and 28.3% for Llama 3.2 3B.

Side by side

Deepseek Coder v2Llama 3.2 3B
ProviderDeepSeekMeta
Noometry Index35.328.4
Released2024-06-172024-09-24
WeightsOpenOpen
Context window—131K
Max output—118K
Input $ / M tokens—$0.05
Output $ / M tokens—$0.33
Results tracked2418

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Deepseek Coder v2 leads

Deepseek Coder v2: 37.5 (#178), Llama 3.2 3B: 27.1 (#314)

BenchmarkDeepseek Coder v2Llama 3.2 3B
BigCodeBench Instruct48.2%23.4%
LMArena Coding12511098
BigCodeBench Complete59.7%28.3%
HumanEval+82.3%—
MBPP+75.1%—

Agentic & Tool Use Not comparable

Deepseek Coder v2: —, Llama 3.2 3B: 19.9 (#135)

BenchmarkDeepseek Coder v2Llama 3.2 3B
Berkeley Function Calling Leaderboard—21.9%
BALROG—10.1%

Reasoning Deepseek Coder v2 leads

Deepseek Coder v2: 22.6 (#175), Llama 3.2 3B: 20.1 (#225)

BenchmarkDeepseek Coder v2Llama 3.2 3B
LMArena Hard Prompts12071095
WinoGrande83.7%—

Math Deepseek Coder v2 leads

Deepseek Coder v2: 32.3 (#188), Llama 3.2 3B: 29.9 (#213)

BenchmarkDeepseek Coder v2Llama 3.2 3B
LMArena Math12411126
GSM8K94.5%—

Knowledge Deepseek Coder v2 leads

Deepseek Coder v2: 31.7 (#209), Llama 3.2 3B: 29.1 (#232)

BenchmarkDeepseek Coder v2Llama 3.2 3B
LMArena Expert11811090
ARC (AI2) Challenge64.3%—

Multilingual Deepseek Coder v2 leads

Deepseek Coder v2: 36.4 (#235), Llama 3.2 3B: 26.4 (#276)

BenchmarkDeepseek Coder v2Llama 3.2 3B
LMArena Non-English11821019
LMArena Chinese12011017
LMArena German11641056
LMArena Russian1188949
LMArena French1184—
LMArena Japanese1126—
LMArena Korean1104—
LMArena Spanish1153—

Instruction Following Deepseek Coder v2 leads

Deepseek Coder v2: 61.9 (#237), Llama 3.2 3B: 56.3 (#270)

BenchmarkDeepseek Coder v2Llama 3.2 3B
LMArena Instruction Following11801089

Long Context Deepseek Coder v2 leads

Deepseek Coder v2: 37.0 (#219), Llama 3.2 3B: 33.4 (#256)

BenchmarkDeepseek Coder v2Llama 3.2 3B
LMArena Longer Query12191101

Writing & Preference Deepseek Coder v2 leads

Deepseek Coder v2: 38.4 (#249), Llama 3.2 3B: 24.9 (#302)

BenchmarkDeepseek Coder v2Llama 3.2 3B
LMArena Text11911110
LMArena Creative Writing11201094
LMArena Multi-Turn11761105
EQ-Bench Creative Writing—595

Frequently asked questions

Is Deepseek Coder v2 better than Llama 3.2 3B?

Deepseek Coder v2 is the stronger model overall, scoring 35.3 to 28.4 on the Noometry Index.

Is Deepseek Coder v2 or Llama 3.2 3B better for coding?

Deepseek Coder v2 scores higher on coding benchmarks: 37.5 versus 27.1 in the Noometry coding category.

How many benchmarks do Deepseek Coder v2 and Llama 3.2 3B share?

15 benchmarks have published results for both models. Deepseek Coder v2 has 24 scored results on Noometry and Llama 3.2 3B has 18.

Related comparisons

Go deeper