Noometry

Model comparison

Codellama 34b Instruct vs Qwen2.5-Coder-32B

Qwen2.5-Coder-32B is the stronger model overall, scoring 33.4 to 30.8 on the Noometry Index.

Last verified . 14 shared benchmarks.

Codellama 34b Instruct Meta

30.8

Rank #287 Confirmed

Qwen2.5-Coder-32B Alibaba (Qwen)

33.4

Rank #245 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Codellama 34b Instruct scores higher in 1 category and Qwen2.5-Coder-32B in 6 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Qwen2.5-Coder-32B leads 41.6 to 28.2.
  • The biggest single-benchmark swing is BigCodeBench Complete: 37.1% for Codellama 34b Instruct and 58% for Qwen2.5-Coder-32B.

Side by side

Codellama 34b InstructQwen2.5-Coder-32B
ProviderMetaAlibaba (Qwen)
Noometry Index30.833.4
Released—2024-09-18
WeightsOpenOpen
Context window—33K
Max output—29K
Input $ / M tokens—$0.66
Output $ / M tokens—$1
Results tracked1431

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Codellama 34b Instruct leads

Codellama 34b Instruct: 28.5 (#314), Qwen2.5-Coder-32B: 22.6 (#333)

BenchmarkCodellama 34b InstructQwen2.5-Coder-32B
BigCodeBench Instruct29%49%
LMArena Coding10461276
BigCodeBench Complete37.1%58%
HumanEval+43.9%87.2%
MBPP+56.3%77%
SWE-bench Verified (bash only)—9%
Aider Polyglot—16.4%
LiveBench Coding—56.9%

Reasoning Qwen2.5-Coder-32B leads

Codellama 34b Instruct: 19.6 (#255), Qwen2.5-Coder-32B: 21.2 (#225)

BenchmarkCodellama 34b InstructQwen2.5-Coder-32B
LMArena Hard Prompts10321251
LiveBench Reasoning—42.1%
LiveBench Data Analysis—49.9%
Epoch Capabilities Index—119.49
HellaSwag—83%
LiveBench—46.2%
WinoGrande—80.8%

Math Qwen2.5-Coder-32B leads

Codellama 34b Instruct: 31.0 (#230), Qwen2.5-Coder-32B: 33.3 (#204)

BenchmarkCodellama 34b InstructQwen2.5-Coder-32B
LMArena Math10561251
LiveBench Math—46.6%
GSM8K—93%

Knowledge Not comparable

Codellama 34b Instruct: —, Qwen2.5-Coder-32B: 33.4 (#203)

BenchmarkCodellama 34b InstructQwen2.5-Coder-32B
LMArena Expert—1221
ARC (AI2) Challenge—70.5%
MMLU—79.1%

Multilingual Qwen2.5-Coder-32B leads

Codellama 34b Instruct: 25.8 (#284), Qwen2.5-Coder-32B: 37.8 (#235)

BenchmarkCodellama 34b InstructQwen2.5-Coder-32B
LMArena Non-English10111205
LMArena Chinese9761222
LMArena Russian—1228

Instruction Following Qwen2.5-Coder-32B leads

Codellama 34b Instruct: 52.2 (#291), Qwen2.5-Coder-32B: 61.4 (#245)

BenchmarkCodellama 34b InstructQwen2.5-Coder-32B
LMArena Instruction Following10281223
LiveBench Instruction Following—58.7%

Long Context Qwen2.5-Coder-32B leads

Codellama 34b Instruct: 30.9 (#284), Qwen2.5-Coder-32B: 38.0 (#208)

BenchmarkCodellama 34b InstructQwen2.5-Coder-32B
LMArena Longer Query10131251

Writing & Preference Qwen2.5-Coder-32B leads

Codellama 34b Instruct: 28.2 (#297), Qwen2.5-Coder-32B: 41.6 (#240)

BenchmarkCodellama 34b InstructQwen2.5-Coder-32B
LMArena Text10661230
LMArena Creative Writing10321174
LMArena Multi-Turn10151222
LiveBench Language—23.3%

Frequently asked questions

Is Codellama 34b Instruct better than Qwen2.5-Coder-32B?

Qwen2.5-Coder-32B is the stronger model overall, scoring 33.4 to 30.8 on the Noometry Index.

Is Codellama 34b Instruct or Qwen2.5-Coder-32B better for coding?

Codellama 34b Instruct scores higher on coding benchmarks: 28.5 versus 22.6 in the Noometry coding category.

How many benchmarks do Codellama 34b Instruct and Qwen2.5-Coder-32B share?

14 benchmarks have published results for both models. Codellama 34b Instruct has 14 scored results on Noometry and Qwen2.5-Coder-32B has 31.

Related comparisons

Go deeper