Noometry

Model comparison

Llama 3.1-70B vs Nemotron 4 340b Instruct

Nemotron 4 340b Instruct is the stronger model overall, scoring 35.3 to 29.4 on the Noometry Index.

Last verified . 17 shared benchmarks.

Llama 3.1-70B Meta

29.4

Rank #299 Confirmed

Nemotron 4 340b Instruct NVIDIA

35.3

Rank #218 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Llama 3.1-70B scores higher in 3 categories and Nemotron 4 340b Instruct in 5 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in math, where Nemotron 4 340b Instruct leads 31.7 to 12.7.

Side by side

Llama 3.1-70BNemotron 4 340b Instruct
ProviderMetaNVIDIA
Noometry Index29.435.3
Released2024-07-232024-06-14
WeightsOpenOpen
Context window128K—
Max output4K—
Input $ / M tokens$0.40—
Output $ / M tokens$0.40—
Results tracked3517

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Nemotron 4 340b Instruct leads

Llama 3.1-70B: 29.8 (#292), Nemotron 4 340b Instruct: 34.6 (#223)

BenchmarkLlama 3.1-70BNemotron 4 340b Instruct
LMArena Coding12601209
WeirdML9%—
BigCodeBench Instruct46.1%—
BigCodeBench Complete54.8%—

Agentic & Tool Use Not comparable

Llama 3.1-70B: 24.9 (#104), Nemotron 4 340b Instruct: —

BenchmarkLlama 3.1-70BNemotron 4 340b Instruct
TheAgentCompany6.9%—
BALROG27.9%—

Reasoning Nemotron 4 340b Instruct leads

Llama 3.1-70B: 20.8 (#215), Nemotron 4 340b Instruct: 22.4 (#180)

BenchmarkLlama 3.1-70BNemotron 4 340b Instruct
LMArena Hard Prompts12411202
DTBench60%—
LMCA14.8%—
Epoch Capabilities Index125.91—

Math Nemotron 4 340b Instruct leads

Llama 3.1-70B: 12.7 (#298), Nemotron 4 340b Instruct: 31.7 (#193)

BenchmarkLlama 3.1-70BNemotron 4 340b Instruct
LMArena Math12521216
OTIS Mock AIME 2024-20253.6%—
Omni-MATH21%—
MATH Level 536.7%—

Knowledge Nemotron 4 340b Instruct leads

Llama 3.1-70B: 24.1 (#264), Nemotron 4 340b Instruct: 31.4 (#212)

BenchmarkLlama 3.1-70BNemotron 4 340b Instruct
LMArena Expert12091171
GPQA Diamond44.2%—
MMLU-Pro65.3%—
GPQA (HELM)42.6%—
MMLU80.1%—

Multilingual Too close to call

Llama 3.1-70B: 38.9 (#220), Nemotron 4 340b Instruct: 38.0 (#228)

BenchmarkLlama 3.1-70BNemotron 4 340b Instruct
LMArena Non-English12191206
LMArena Chinese12151214
LMArena French12601196
LMArena German12221192
LMArena Japanese11321128
LMArena Korean11401148
LMArena Russian12341219
LMArena Spanish12531195

Instruction Following Llama 3.1-70B leads

Llama 3.1-70B: 65.5 (#218), Nemotron 4 340b Instruct: 63.3 (#227)

BenchmarkLlama 3.1-70BNemotron 4 340b Instruct
LMArena Instruction Following12311203
IFEval82.1%—

Long Context Too close to call

Llama 3.1-70B: 37.6 (#208), Nemotron 4 340b Instruct: 37.1 (#216)

BenchmarkLlama 3.1-70BNemotron 4 340b Instruct
LMArena Longer Query12411225

Writing & Preference Nemotron 4 340b Instruct leads

Llama 3.1-70B: 35.6 (#262), Nemotron 4 340b Instruct: 42.9 (#229)

BenchmarkLlama 3.1-70BNemotron 4 340b Instruct
LMArena Text12611225
LMArena Creative Writing12321203
LMArena Multi-Turn12561214
EQ-Bench Creative Writing784—
WildBench75.8%—

Frequently asked questions

Is Llama 3.1-70B better than Nemotron 4 340b Instruct?

Nemotron 4 340b Instruct is the stronger model overall, scoring 35.3 to 29.4 on the Noometry Index.

Is Llama 3.1-70B or Nemotron 4 340b Instruct better for coding?

Nemotron 4 340b Instruct scores higher on coding benchmarks: 34.6 versus 29.8 in the Noometry coding category.

How many benchmarks do Llama 3.1-70B and Nemotron 4 340b Instruct share?

17 benchmarks have published results for both models. Llama 3.1-70B has 35 scored results on Noometry and Nemotron 4 340b Instruct has 17.

Related comparisons

Go deeper