Noometry

Model comparison

Llama 3-70B vs Mistral Large

Mistral Large is the stronger model overall, scoring 31.9 to 28.8 on the Noometry Index.

Last verified . 28 shared benchmarks.

Llama 3-70B Meta

28.8

Rank #323 Confirmed

Mistral Large Mistral AI

31.9

Rank #263 Confirmed

Summary

  • They share 28 benchmarks with published results for both. Llama 3-70B scores higher in 3 categories and Mistral Large in 6 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Mistral Large leads 30.1 to 20.8.
  • The biggest single-benchmark swing is MATH Level 5: 22.6% for Llama 3-70B and 50.3% for Mistral Large.

Side by side

Llama 3-70BMistral Large
ProviderMetaMistral AI
Noometry Index28.831.9
Released2024-04-182024-02-26
WeightsOpenOpen
Context window—131K
Max output—16K
Input $ / M tokens—$2
Output $ / M tokens—$6
Results tracked3151

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3-70B leads

Llama 3-70B: 35.8 (#218), Mistral Large: 34.3 (#240)

BenchmarkLlama 3-70BMistral Large
BigCodeBench Instruct43.6%30%
LMArena Coding12061277
BigCodeBench Complete54.5%38.3%
HumanEval+72%62.2%
MBPP+69%59.5%
SciCode—36.2%
LiveBench Coding—47.1%
ALE-Bench—264.7

Agentic & Tool Use Mistral Large leads

Llama 3-70B: 21.1 (#139), Mistral Large: 28.6 (#89)

BenchmarkLlama 3-70BMistral Large
Berkeley Function Calling Leaderboard—38.4%
Cybench5%—

Reasoning Llama 3-70B leads

Llama 3-70B: 18.0 (#288), Mistral Large: 15.8 (#310)

BenchmarkLlama 3-70BMistral Large
LMArena Hard Prompts11951257
DTBench54.2%65.1%
Epoch Capabilities Index122.93128.52
ForecastBench57.157.1
SimpleBench—22.5%
Kagi LLM Benchmark35.1%—
CritPt—0%
LiveBench Reasoning—43.5%
LiveBench Data Analysis—50.1%
LMCA—16.7%
LiveBench—48.4%
WinoGrande83.5%—

Math Mistral Large leads

Llama 3-70B: 12.8 (#305), Mistral Large: 18.2 (#291)

BenchmarkLlama 3-70BMistral Large
OTIS Mock AIME 2024-20254.3%8.5%
LMArena Math12181262
MATH Level 522.6%50.3%
Omni-MATH—28.1%
LiveBench Math—42.5%
FrontierMath (Feb 2025 set)—0.3%

Knowledge Mistral Large leads

Llama 3-70B: 20.8 (#277), Mistral Large: 30.1 (#230)

BenchmarkLlama 3-70BMistral Large
GPQA Diamond40.6%51.3%
LMArena Expert11491232
MMLU79.3%80%
MMLU-Pro—59.9%
Confabulations—21.4%
Vectara Hallucination Rate—4.5%
GPQA (HELM)—43.5%

Multilingual Mistral Large leads

Llama 3-70B: 33.6 (#251), Mistral Large: 40.0 (#219)

BenchmarkLlama 3-70BMistral Large
LMArena Non-English11421237
LMArena Chinese11141240
LMArena French12321325
LMArena German11691254
LMArena Japanese10171188
LMArena Korean10171202
LMArena Russian11591257
LMArena Spanish12411268

Instruction Following Mistral Large leads

Llama 3-70B: 62.5 (#238), Mistral Large: 67.9 (#191)

BenchmarkLlama 3-70BMistral Large
LMArena Instruction Following11941249
LiveBench Instruction Following—67.9%
IFEval—87.7%

Long Context Mistral Large leads

Llama 3-70B: 35.6 (#240), Mistral Large: 38.3 (#199)

BenchmarkLlama 3-70BMistral Large
LMArena Longer Query11741261

Writing & Preference Llama 3-70B leads

Llama 3-70B: 42.8 (#231), Mistral Large: 40.7 (#242)

BenchmarkLlama 3-70BMistral Large
LMArena Text12211266
LMArena Creative Writing12101243
LMArena Multi-Turn12231260
Short-Story Creative Writing—69%
EQ-Bench Creative Writing—985
WildBench—80.1%
LiveBench Language—39.4%

Frequently asked questions

Is Llama 3-70B better than Mistral Large?

Mistral Large is the stronger model overall, scoring 31.9 to 28.8 on the Noometry Index.

Is Llama 3-70B or Mistral Large better for coding?

Llama 3-70B scores higher on coding benchmarks: 35.8 versus 34.3 in the Noometry coding category.

How many benchmarks do Llama 3-70B and Mistral Large share?

28 benchmarks have published results for both models. Llama 3-70B has 31 scored results on Noometry and Mistral Large has 51.

Related comparisons

Go deeper