Noometry

Model comparison

Nemotron 4 340b Instruct vs Qwen2.5-Coder-32B

Nemotron 4 340b Instruct is the stronger model overall, scoring 35.3 to 32.9 on the Noometry Index.

Last verified . 12 shared benchmarks.

Nemotron 4 340b Instruct NVIDIA

35.3

Rank #218 Confirmed

Qwen2.5-Coder-32B Alibaba (Qwen)

32.9

Rank #239 Confirmed

Summary

  • They share 12 benchmarks with published results for both. Nemotron 4 340b Instruct scores higher in 6 categories and Qwen2.5-Coder-32B in 2 categories; 5 gaps are clear of the uncertainty.
  • The widest gap is in coding, where Nemotron 4 340b Instruct leads 34.6 to 22.4.

Side by side

Nemotron 4 340b InstructQwen2.5-Coder-32B
ProviderNVIDIAAlibaba (Qwen)
Noometry Index35.332.9
Released2024-06-142024-09-18
WeightsOpenOpen
Context window—33K
Max output—29K
Input $ / M tokens—$0.66
Output $ / M tokens—$1
Results tracked1731

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Nemotron 4 340b Instruct leads

Nemotron 4 340b Instruct: 34.6 (#223), Qwen2.5-Coder-32B: 22.4 (#328)

BenchmarkNemotron 4 340b InstructQwen2.5-Coder-32B
LMArena Coding12091276
SWE-bench Verified (bash only)—9%
Aider Polyglot—16.4%
BigCodeBench Instruct—49%
LiveBench Coding—56.9%
BigCodeBench Complete—58%
HumanEval+—87.2%
MBPP+—77%

Reasoning Nemotron 4 340b Instruct leads

Nemotron 4 340b Instruct: 22.4 (#180), Qwen2.5-Coder-32B: 20.4 (#220)

BenchmarkNemotron 4 340b InstructQwen2.5-Coder-32B
LMArena Hard Prompts12021251
LiveBench Reasoning—42.1%
LiveBench Data Analysis—49.9%
Epoch Capabilities Index—119.41
HellaSwag—83%
LiveBench—46.2%
WinoGrande—80.8%

Math Too close to call

Nemotron 4 340b Instruct: 31.7 (#193), Qwen2.5-Coder-32B: 30.8 (#203)

BenchmarkNemotron 4 340b InstructQwen2.5-Coder-32B
LMArena Math12161251
LiveBench Math—46.6%
GSM8K—93%

Knowledge Qwen2.5-Coder-32B leads

Nemotron 4 340b Instruct: 31.4 (#212), Qwen2.5-Coder-32B: 32.9 (#200)

BenchmarkNemotron 4 340b InstructQwen2.5-Coder-32B
LMArena Expert11711221
ARC (AI2) Challenge—70.5%
MMLU—79.1%

Multilingual Too close to call

Nemotron 4 340b Instruct: 38.0 (#228), Qwen2.5-Coder-32B: 38.0 (#230)

BenchmarkNemotron 4 340b InstructQwen2.5-Coder-32B
LMArena Non-English12061205
LMArena Chinese12141222
LMArena Russian12191228
LMArena French1196—
LMArena German1192—
LMArena Japanese1128—
LMArena Korean1148—
LMArena Spanish1195—

Instruction Following Nemotron 4 340b Instruct leads

Nemotron 4 340b Instruct: 63.3 (#227), Qwen2.5-Coder-32B: 61.6 (#240)

BenchmarkNemotron 4 340b InstructQwen2.5-Coder-32B
LMArena Instruction Following12031223
LiveBench Instruction Following—58.7%

Long Context Too close to call

Nemotron 4 340b Instruct: 37.1 (#216), Qwen2.5-Coder-32B: 37.9 (#204)

BenchmarkNemotron 4 340b InstructQwen2.5-Coder-32B
LMArena Longer Query12251251

Writing & Preference Nemotron 4 340b Instruct leads

Nemotron 4 340b Instruct: 42.9 (#229), Qwen2.5-Coder-32B: 41.8 (#235)

BenchmarkNemotron 4 340b InstructQwen2.5-Coder-32B
LMArena Text12251230
LMArena Creative Writing12031174
LMArena Multi-Turn12141222
LiveBench Language—23.3%

Frequently asked questions

Is Nemotron 4 340b Instruct better than Qwen2.5-Coder-32B?

Nemotron 4 340b Instruct is the stronger model overall, scoring 35.3 to 32.9 on the Noometry Index.

Is Nemotron 4 340b Instruct or Qwen2.5-Coder-32B better for coding?

Nemotron 4 340b Instruct scores higher on coding benchmarks: 34.6 versus 22.4 in the Noometry coding category.

How many benchmarks do Nemotron 4 340b Instruct and Qwen2.5-Coder-32B share?

12 benchmarks have published results for both models. Nemotron 4 340b Instruct has 17 scored results on Noometry and Qwen2.5-Coder-32B has 31.

Related comparisons

Go deeper