Noometry

Model comparison

Llama 3.1 Nemotron 70b Instruct vs Phi 3 Medium 4k Instruct

Llama 3.1 Nemotron 70b Instruct is the stronger model overall, scoring 37.0 to 32.4 on the Noometry Index.

Last verified . 12 shared benchmarks.

Llama 3.1 Nemotron 70b Instruct NVIDIA

37.0

Rank #199 Confirmed

Phi 3 Medium 4k Instruct Microsoft

32.4

Rank #246 Confirmed

Summary

  • They share 12 benchmarks with published results for both. Llama 3.1 Nemotron 70b Instruct scores higher in 8 categories and Phi 3 Medium 4k Instruct in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Llama 3.1 Nemotron 70b Instruct leads 48.6 to 34.3.

Side by side

Llama 3.1 Nemotron 70b InstructPhi 3 Medium 4k Instruct
ProviderNVIDIAMicrosoft
Noometry Index37.032.4
Released2024-12-18—
WeightsOpenOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked1417

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 35.3 (#212), Phi 3 Medium 4k Instruct: 32.3 (#263)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Medium 4k Instruct
LMArena Coding12721130
BigCodeBench Instruct38.7%—
BigCodeBench Complete48.2%—

Reasoning Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 24.0 (#156), Phi 3 Medium 4k Instruct: 20.8 (#214)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Medium 4k Instruct
LMArena Hard Prompts12661127

Math Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 32.9 (#182), Phi 3 Medium 4k Instruct: 30.9 (#202)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Medium 4k Instruct
LMArena Math12711173

Knowledge Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 33.5 (#194), Phi 3 Medium 4k Instruct: 29.6 (#226)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Medium 4k Instruct
LMArena Expert12421108

Multilingual Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 40.7 (#212), Phi 3 Medium 4k Instruct: 30.8 (#258)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Medium 4k Instruct
LMArena Non-English12451094
LMArena Chinese12631108
LMArena Russian12271145
LMArena French—1109
LMArena German—1101
LMArena Japanese—1042
LMArena Korean—954
LMArena Spanish—1095

Instruction Following Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 66.2 (#208), Phi 3 Medium 4k Instruct: 57.8 (#263)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Medium 4k Instruct
LMArena Instruction Following12521114

Long Context Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 37.5 (#210), Phi 3 Medium 4k Instruct: 33.9 (#250)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Medium 4k Instruct
LMArena Longer Query12391120

Writing & Preference Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 48.6 (#197), Phi 3 Medium 4k Instruct: 34.3 (#267)

BenchmarkLlama 3.1 Nemotron 70b InstructPhi 3 Medium 4k Instruct
LMArena Text12831138
LMArena Creative Writing12691106
LMArena Multi-Turn12751088

Frequently asked questions

Is Llama 3.1 Nemotron 70b Instruct better than Phi 3 Medium 4k Instruct?

Llama 3.1 Nemotron 70b Instruct is the stronger model overall, scoring 37.0 to 32.4 on the Noometry Index.

Is Llama 3.1 Nemotron 70b Instruct or Phi 3 Medium 4k Instruct better for coding?

Llama 3.1 Nemotron 70b Instruct scores higher on coding benchmarks: 35.3 versus 32.3 in the Noometry coding category.

How many benchmarks do Llama 3.1 Nemotron 70b Instruct and Phi 3 Medium 4k Instruct share?

12 benchmarks have published results for both models. Llama 3.1 Nemotron 70b Instruct has 14 scored results on Noometry and Phi 3 Medium 4k Instruct has 17.

Related comparisons

Go deeper