Noometry

Model comparison

Llama 3.1-70B vs Ministral 3B

Llama 3.1-70B is the stronger model overall, scoring 29.6 to 26.2 on the Noometry Index. Ministral 3B costs 4.0× less per token, which makes it the better buy when Llama 3.1-70B's lead doesn't matter for your workload.

Last verified . 5 shared benchmarks.

Llama 3.1-70B Meta

29.6

Rank #308 Confirmed

Ministral 3B Mistral AI

26.2

Rank #338 Confirmed

Summary

  • They share 5 benchmarks with published results for both. Llama 3.1-70B scores higher in 2 categories and Ministral 3B in 1 category; 3 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Llama 3.1-70B leads 24.2 to 10.4.
  • The biggest single-benchmark swing is MATH Level 5: 36.7% for Llama 3.1-70B and 14.4% for Ministral 3B.
  • Ministral 3B is cheaper at $0.10 / $0.10 per million input/output tokens, against $0.40 / $0.40 for Llama 3.1-70B.
  • Ministral 3B accepts more context: 131K tokens versus 128K.

Side by side

Llama 3.1-70BMinistral 3B
ProviderMetaMistral AI
Noometry Index29.626.2
Released2024-07-232024-10-01
WeightsOpenOpen
Context window128K131K
Max output4K262K
Input $ / M tokens$0.40$0.10
Output $ / M tokens$0.40$0.10
Results tracked356

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Llama 3.1-70B: 30.3 (#296), Ministral 3B: —

BenchmarkLlama 3.1-70BMinistral 3B
WeirdML9%—
BigCodeBench Instruct46.1%—
LMArena Coding1260—
BigCodeBench Complete54.8%—

Agentic & Tool Use Not comparable

Llama 3.1-70B: 25.1 (#112), Ministral 3B: —

BenchmarkLlama 3.1-70BMinistral 3B
TheAgentCompany6.9%—
BALROG27.9%—

Reasoning Llama 3.1-70B leads

Llama 3.1-70B: 21.6 (#220), Ministral 3B: 18.4 (#282)

BenchmarkLlama 3.1-70BMinistral 3B
DTBench60%51.7%
LMCA14.8%5.5%
Epoch Capabilities Index125.92118.1
LMArena Hard Prompts1241—

Math Ministral 3B leads

Llama 3.1-70B: 13.5 (#304), Ministral 3B: 26.6 (#258)

BenchmarkLlama 3.1-70BMinistral 3B
MATH Level 536.7%14.4%
OTIS Mock AIME 2024-20253.6%—
Omni-MATH21%—
LMArena Math1252—

Knowledge Llama 3.1-70B leads

Llama 3.1-70B: 24.2 (#269), Ministral 3B: 10.4 (#302)

BenchmarkLlama 3.1-70BMinistral 3B
GPQA Diamond44.2%25.3%
MMLU-Pro65.3%—
Vectara Hallucination Rate—7.3%
GPQA (HELM)42.6%—
LMArena Expert1209—
MMLU80.1%—

Multilingual Not comparable

Llama 3.1-70B: 38.8 (#225), Ministral 3B: —

BenchmarkLlama 3.1-70BMinistral 3B
LMArena Non-English1219—
LMArena Chinese1215—
LMArena French1261—
LMArena German1222—
LMArena Japanese1132—
LMArena Korean1140—
LMArena Russian1234—
LMArena Spanish1253—

Instruction Following Not comparable

Llama 3.1-70B: 65.3 (#223), Ministral 3B: —

BenchmarkLlama 3.1-70BMinistral 3B
IFEval82.1%—
LMArena Instruction Following1231—

Long Context Not comparable

Llama 3.1-70B: 37.6 (#214), Ministral 3B: —

BenchmarkLlama 3.1-70BMinistral 3B
LMArena Longer Query1241—

Writing & Preference Not comparable

Llama 3.1-70B: 35.4 (#267), Ministral 3B: —

BenchmarkLlama 3.1-70BMinistral 3B
LMArena Text1261—
LMArena Creative Writing1232—
EQ-Bench Creative Writing784—
WildBench75.8%—
LMArena Multi-Turn1256—

Frequently asked questions

Is Llama 3.1-70B better than Ministral 3B?

Llama 3.1-70B is the stronger model overall, scoring 29.6 to 26.2 on the Noometry Index. Ministral 3B costs 4.0× less per token, which makes it the better buy when Llama 3.1-70B's lead doesn't matter for your workload.

Which is cheaper, Llama 3.1-70B or Ministral 3B?

Ministral 3B is cheaper. It lists at $0.10 per million input tokens and $0.10 per million output tokens; Llama 3.1-70B lists at $0.40 and $0.40.

Which has the bigger context window?

Ministral 3B does, with 131K tokens against 128K.

How many benchmarks do Llama 3.1-70B and Ministral 3B share?

5 benchmarks have published results for both models. Llama 3.1-70B has 35 scored results on Noometry and Ministral 3B has 6.

Related comparisons

Go deeper