Noometry

Model comparison

Llama 3.1 Nemotron 70b Instruct vs Phi 3 Mini 128k Instruct

Llama 3.1 Nemotron 70b Instruct is the stronger model overall, scoring 37.6 to 29.7 on the Noometry Index.

Last verified . 14 shared benchmarks.

Llama 3.1 Nemotron 70b Instruct NVIDIA

37.6

Rank #203 Confirmed

Phi 3 Mini 128k Instruct Microsoft

29.7

Rank #305 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Llama 3.1 Nemotron 70b Instruct scores higher in 8 categories and Phi 3 Mini 128k Instruct in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Llama 3.1 Nemotron 70b Instruct leads 48.4 to 27.1.
  • The biggest single-benchmark swing is BigCodeBench Instruct: 38.7% for Llama 3.1 Nemotron 70b Instruct and 29.6% for Phi 3 Mini 128k Instruct.

Side by side

Llama 3.1 Nemotron 70b InstructPhi 3 Mini 128k Instruct
ProviderNVIDIAMicrosoft
Noometry Index37.629.7
Released2024-12-182024-04-23
WeightsOpenOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked1419

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 35.9 (#216), Phi 3 Mini 128k Instruct: 28.8 (#312)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Mini 128k Instruct
BigCodeBench Instruct38.7%29.6%
LMArena Coding12721039
BigCodeBench Complete48.2%40.6%

Reasoning Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 25.0 (#152), Phi 3 Mini 128k Instruct: 19.6 (#256)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Mini 128k Instruct
LMArena Hard Prompts12661028

Math Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 35.5 (#182), Phi 3 Mini 128k Instruct: 31.6 (#222)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Mini 128k Instruct
LMArena Math12711089

Knowledge Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 34.1 (#199), Phi 3 Mini 128k Instruct: 26.8 (#254)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Mini 128k Instruct
LMArena Expert1242984

Multilingual Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 40.5 (#217), Phi 3 Mini 128k Instruct: 25.2 (#285)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Mini 128k Instruct
LMArena Non-English12451000
LMArena Chinese12631016
LMArena Russian12271004
LMArena French—1039
LMArena German—1006
LMArena Japanese—899
LMArena Korean—856
LMArena Spanish—1059

Instruction Following Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 65.9 (#213), Phi 3 Mini 128k Instruct: 51.8 (#294)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Mini 128k Instruct
LMArena Instruction Following12521023

Long Context Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 37.6 (#215), Phi 3 Mini 128k Instruct: 30.4 (#289)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Mini 128k Instruct
LMArena Longer Query1238996

Writing & Preference Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 48.4 (#203), Phi 3 Mini 128k Instruct: 27.1 (#301)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Mini 128k Instruct
LMArena Text12831050
LMArena Creative Writing12691024
LMArena Multi-Turn1275989

Frequently asked questions

Is Llama 3.1 Nemotron 70b Instruct better than Phi 3 Mini 128k Instruct?

Llama 3.1 Nemotron 70b Instruct is the stronger model overall, scoring 37.6 to 29.7 on the Noometry Index.

Is Llama 3.1 Nemotron 70b Instruct or Phi 3 Mini 128k Instruct better for coding?

Llama 3.1 Nemotron 70b Instruct scores higher on coding benchmarks: 35.9 versus 28.8 in the Noometry coding category.

How many benchmarks do Llama 3.1 Nemotron 70b Instruct and Phi 3 Mini 128k Instruct share?

14 benchmarks have published results for both models. Llama 3.1 Nemotron 70b Instruct has 14 scored results on Noometry and Phi 3 Mini 128k Instruct has 19.

Related comparisons

Go deeper