Noometry

Model comparison

Llama 3-70B vs Qwen2.5-Coder-32B

Qwen2.5-Coder-32B is the stronger model overall, scoring 33.4 to 28.8 on the Noometry Index.

Last verified . 19 shared benchmarks.

Llama 3-70B Meta

28.8

Rank #323 Confirmed

Qwen2.5-Coder-32B Alibaba (Qwen)

33.4

Rank #245 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Llama 3-70B scores higher in 3 categories and Qwen2.5-Coder-32B in 5 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Qwen2.5-Coder-32B leads 33.3 to 12.8.
  • The biggest single-benchmark swing is BigCodeBench Instruct: 43.6% for Llama 3-70B and 49% for Qwen2.5-Coder-32B.

Side by side

Llama 3-70BQwen2.5-Coder-32B
ProviderMetaAlibaba (Qwen)
Noometry Index28.833.4
Released2024-04-182024-09-18
WeightsOpenOpen
Context window—33K
Max output—29K
Input $ / M tokens—$0.66
Output $ / M tokens—$1
Results tracked3131

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3-70B leads

Llama 3-70B: 35.8 (#218), Qwen2.5-Coder-32B: 22.6 (#333)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
BigCodeBench Instruct43.6%49%
LMArena Coding12061276
BigCodeBench Complete54.5%58%
HumanEval+72%87.2%
MBPP+69%77%
SWE-bench Verified (bash only)—9%
Aider Polyglot—16.4%
LiveBench Coding—56.9%

Agentic & Tool Use Not comparable

Llama 3-70B: 21.1 (#139), Qwen2.5-Coder-32B: —

BenchmarkLlama 3-70BQwen2.5-Coder-32B
Cybench5%—

Reasoning Qwen2.5-Coder-32B leads

Llama 3-70B: 18.0 (#288), Qwen2.5-Coder-32B: 21.2 (#225)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Hard Prompts11951251
Epoch Capabilities Index122.93119.49
WinoGrande83.5%80.8%
Kagi LLM Benchmark35.1%—
LiveBench Reasoning—42.1%
DTBench54.2%—
LiveBench Data Analysis—49.9%
ForecastBench57.1—
HellaSwag—83%
LiveBench—46.2%

Math Qwen2.5-Coder-32B leads

Llama 3-70B: 12.8 (#305), Qwen2.5-Coder-32B: 33.3 (#204)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Math12181251
OTIS Mock AIME 2024-20254.3%—
LiveBench Math—46.6%
MATH Level 522.6%—
GSM8K—93%

Knowledge Qwen2.5-Coder-32B leads

Llama 3-70B: 20.8 (#277), Qwen2.5-Coder-32B: 33.4 (#203)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Expert11491221
MMLU79.3%79.1%
GPQA Diamond40.6%—
ARC (AI2) Challenge—70.5%

Multilingual Qwen2.5-Coder-32B leads

Llama 3-70B: 33.6 (#251), Qwen2.5-Coder-32B: 37.8 (#235)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Non-English11421205
LMArena Chinese11141222
LMArena Russian11591228
LMArena French1232—
LMArena German1169—
LMArena Japanese1017—
LMArena Korean1017—
LMArena Spanish1241—

Instruction Following Llama 3-70B leads

Llama 3-70B: 62.5 (#238), Qwen2.5-Coder-32B: 61.4 (#245)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Instruction Following11941223
LiveBench Instruction Following—58.7%

Long Context Qwen2.5-Coder-32B leads

Llama 3-70B: 35.6 (#240), Qwen2.5-Coder-32B: 38.0 (#208)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Longer Query11741251

Writing & Preference Llama 3-70B leads

Llama 3-70B: 42.8 (#231), Qwen2.5-Coder-32B: 41.6 (#240)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Text12211230
LMArena Creative Writing12101174
LMArena Multi-Turn12231222
LiveBench Language—23.3%

Frequently asked questions

Is Llama 3-70B better than Qwen2.5-Coder-32B?

Qwen2.5-Coder-32B is the stronger model overall, scoring 33.4 to 28.8 on the Noometry Index.

Is Llama 3-70B or Qwen2.5-Coder-32B better for coding?

Llama 3-70B scores higher on coding benchmarks: 35.8 versus 22.6 in the Noometry coding category.

How many benchmarks do Llama 3-70B and Qwen2.5-Coder-32B share?

19 benchmarks have published results for both models. Llama 3-70B has 31 scored results on Noometry and Qwen2.5-Coder-32B has 31.

Related comparisons

Go deeper