Noometry

Model comparison

Llama 3.1-70B vs Mistral Small 3

Mistral Small 3 is the stronger model overall, scoring 31.2 to 29.6 on the Noometry Index.

Last verified . 22 shared benchmarks.

Llama 3.1-70B Meta

29.6

Rank #308 Confirmed

Mistral Small 3 Mistral AI

31.2

Rank #278 Confirmed

Summary

  • They share 22 benchmarks with published results for both. Llama 3.1-70B scores higher in 4 categories and Mistral Small 3 in 4 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in coding, where Mistral Small 3 leads 36.5 to 30.3.
  • Mistral Small 3 is cheaper at $0.05 / $0.08 per million input/output tokens, against $0.40 / $0.40 for Llama 3.1-70B.
  • Llama 3.1-70B accepts more context: 128K tokens versus 33K.

Side by side

Llama 3.1-70BMistral Small 3
ProviderMetaMistral AI
Noometry Index29.631.2
Released2024-07-232025-01-30
WeightsOpenOpen
Context window128K33K
Max output4K16K
Input $ / M tokens$0.40$0.05
Output $ / M tokens$0.40$0.08
Results tracked3524

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Small 3 leads

Llama 3.1-70B: 30.3 (#296), Mistral Small 3: 36.5 (#207)

BenchmarkLlama 3.1-70BMistral Small 3
BigCodeBench Instruct46.1%45.3%
LMArena Coding12601246
BigCodeBench Complete54.8%50.4%
WeirdML9%—

Agentic & Tool Use Not comparable

Llama 3.1-70B: 25.1 (#112), Mistral Small 3: —

BenchmarkLlama 3.1-70BMistral Small 3
TheAgentCompany6.9%—
BALROG27.9%—

Reasoning Llama 3.1-70B leads

Llama 3.1-70B: 21.6 (#220), Mistral Small 3: 18.9 (#273)

BenchmarkLlama 3.1-70BMistral Small 3
LMArena Hard Prompts12411233
Epoch Capabilities Index125.92127.07
Chess Puzzles—0%
DTBench60%—
LMCA14.8%—

Math Mistral Small 3 leads

Llama 3.1-70B: 13.5 (#304), Mistral Small 3: 16.3 (#295)

BenchmarkLlama 3.1-70BMistral Small 3
OTIS Mock AIME 2024-20253.6%6.7%
LMArena Math12521240
Omni-MATH21%—
MATH Level 536.7%—

Knowledge Too close to call

Llama 3.1-70B: 24.2 (#269), Mistral Small 3: 25.1 (#263)

BenchmarkLlama 3.1-70BMistral Small 3
GPQA Diamond44.2%47.3%
LMArena Expert12091202
MMLU-Pro65.3%—
Confabulations—25.2%
GPQA (HELM)42.6%—
MMLU80.1%—

Multilingual Llama 3.1-70B leads

Llama 3.1-70B: 38.8 (#225), Mistral Small 3: 37.3 (#236)

BenchmarkLlama 3.1-70BMistral Small 3
LMArena Non-English12191198
LMArena Chinese12151204
LMArena French12611203
LMArena German12221211
LMArena Japanese11321111
LMArena Korean11401188
LMArena Russian12341216
LMArena Spanish1253—

Instruction Following Llama 3.1-70B leads

Llama 3.1-70B: 65.3 (#223), Mistral Small 3: 63.7 (#229)

BenchmarkLlama 3.1-70BMistral Small 3
LMArena Instruction Following12311214
IFEval82.1%—

Long Context Too close to call

Llama 3.1-70B: 37.6 (#214), Mistral Small 3: 37.8 (#211)

BenchmarkLlama 3.1-70BMistral Small 3
LMArena Longer Query12411246

Writing & Preference Llama 3.1-70B leads

Llama 3.1-70B: 35.4 (#267), Mistral Small 3: 32.2 (#280)

BenchmarkLlama 3.1-70BMistral Small 3
LMArena Text12611234
LMArena Creative Writing12321195
EQ-Bench Creative Writing784707
LMArena Multi-Turn12561217
WildBench75.8%—

Frequently asked questions

Is Llama 3.1-70B better than Mistral Small 3?

Mistral Small 3 is the stronger model overall, scoring 31.2 to 29.6 on the Noometry Index.

Which is cheaper, Llama 3.1-70B or Mistral Small 3?

Mistral Small 3 is cheaper. It lists at $0.05 per million input tokens and $0.08 per million output tokens; Llama 3.1-70B lists at $0.40 and $0.40.

Is Llama 3.1-70B or Mistral Small 3 better for coding?

Mistral Small 3 scores higher on coding benchmarks: 36.5 versus 30.3 in the Noometry coding category.

Which has the bigger context window?

Llama 3.1-70B does, with 128K tokens against 33K.

How many benchmarks do Llama 3.1-70B and Mistral Small 3 share?

22 benchmarks have published results for both models. Llama 3.1-70B has 35 scored results on Noometry and Mistral Small 3 has 24.

Related comparisons

Go deeper