Noometry

Model comparison

Llama 3.1 Nemotron 70b Instruct vs o1-mini

Llama 3.1 Nemotron 70b Instruct is the stronger model overall, scoring 37.6 to 34.0 on the Noometry Index.

Last verified . 12 shared benchmarks.

Llama 3.1 Nemotron 70b Instruct NVIDIA

37.6

Rank #203 Confirmed

o1-mini OpenAI

34.0

Rank #235 Confirmed

Summary

  • They share 12 benchmarks with published results for both. Llama 3.1 Nemotron 70b Instruct scores higher in 3 categories and o1-mini in 5 categories; 3 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Llama 3.1 Nemotron 70b Instruct leads 25.0 to 8.8.
  • Llama 3.1 Nemotron 70b Instruct has downloadable open weights; the other is API-only.

Side by side

Llama 3.1 Nemotron 70b Instructo1-mini
ProviderNVIDIAOpenAI
Noometry Index37.634.0
Released2024-12-182024-09-12
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked1439

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

Llama 3.1 Nemotron 70b Instruct: 35.9 (#216), o1-mini: 35.5 (#224)

BenchmarkLlama 3.1 Nemotron 70b Instructo1-mini
LMArena Coding12721362
Aider Polyglot—32.9%
WeirdML—36.3%
BigCodeBench Instruct38.7%—
LiveBench Coding—48%
BigCodeBench Complete48.2%—
HumanEval+—89%
MBPP+—78.8%

Agentic & Tool Use Not comparable

Llama 3.1 Nemotron 70b Instruct: —, o1-mini: 24.6 (#118)

BenchmarkLlama 3.1 Nemotron 70b Instructo1-mini
Cybench—10%

Reasoning Llama 3.1 Nemotron 70b Instruct leads

Llama 3.1 Nemotron 70b Instruct: 25.0 (#152), o1-mini: 8.8 (#346)

BenchmarkLlama 3.1 Nemotron 70b Instructo1-mini
LMArena Hard Prompts12661333
ARC-AGI-2—0.8%
SimpleBench—18.1%
ARC-AGI-1—14%
LiveBench Reasoning—72.3%
LiveBench Data Analysis—57.9%
Epoch Capabilities Index—135.82
LiveBench—57.8%

Math Too close to call

Llama 3.1 Nemotron 70b Instruct: 35.5 (#182), o1-mini: 35.4 (#186)

BenchmarkLlama 3.1 Nemotron 70b Instructo1-mini
LMArena Math12711358
OTIS Mock AIME 2024-2025—46.9%
LiveBench Math—62%
MATH Level 5—89.2%
FrontierMath (Feb 2025 set)—1.7%

Knowledge Too close to call

Llama 3.1 Nemotron 70b Instruct: 34.1 (#199), o1-mini: 34.9 (#192)

BenchmarkLlama 3.1 Nemotron 70b Instructo1-mini
LMArena Expert12421316
GPQA Diamond—62.4%
Confabulations—18.6%

Multilingual o1-mini leads

Llama 3.1 Nemotron 70b Instruct: 40.5 (#217), o1-mini: 43.6 (#182)

BenchmarkLlama 3.1 Nemotron 70b Instructo1-mini
LMArena Non-English12451289
LMArena Chinese12631314
LMArena Russian12271283
LMArena French—1293
LMArena German—1278
LMArena Japanese—1245
LMArena Korean—1223
LMArena Spanish—1303

Instruction Following Too close to call

Llama 3.1 Nemotron 70b Instruct: 65.9 (#213), o1-mini: 66.7 (#206)

BenchmarkLlama 3.1 Nemotron 70b Instructo1-mini
LMArena Instruction Following12521304
LiveBench Instruction Following—65.4%

Long Context o1-mini leads

Llama 3.1 Nemotron 70b Instruct: 37.6 (#215), o1-mini: 40.1 (#161)

BenchmarkLlama 3.1 Nemotron 70b Instructo1-mini
LMArena Longer Query12381320

Writing & Preference Too close to call

Llama 3.1 Nemotron 70b Instruct: 48.4 (#203), o1-mini: 48.4 (#202)

BenchmarkLlama 3.1 Nemotron 70b Instructo1-mini
LMArena Text12831317
LMArena Creative Writing12691244
LMArena Multi-Turn12751314
Short-Story Creative Writing—64.9%
LiveBench Language—40.9%

Frequently asked questions

Is Llama 3.1 Nemotron 70b Instruct better than o1-mini?

Llama 3.1 Nemotron 70b Instruct is the stronger model overall, scoring 37.6 to 34.0 on the Noometry Index.

Is Llama 3.1 Nemotron 70b Instruct or o1-mini better for coding?

They score almost the same on coding (35.9 vs 35.5); test both on your own repository before choosing.

How many benchmarks do Llama 3.1 Nemotron 70b Instruct and o1-mini share?

12 benchmarks have published results for both models. Llama 3.1 Nemotron 70b Instruct has 14 scored results on Noometry and o1-mini has 39.

Related comparisons

Go deeper