Model comparison
Llama 13b vs Ministral 3B
Ministral 3B is the stronger model overall, scoring 26.2 to 24.4 on the Noometry Index.
Last verified . 1 shared benchmarks.
Meta
Mistral AISummary
- They share 1 benchmark with published results for both. Llama 13b scores higher in 1 category and Ministral 3B in 1 category; one gap is clear of the uncertainty.
- The widest gap is in reasoning, where Ministral 3B leads 18.4 to 14.0.
Side by side
| Llama 13b | Ministral 3B | |
|---|---|---|
| Provider | Meta | Mistral AI |
| Noometry Index | 24.4 | 26.2 |
| Released | 2023-02-24 | 2024-10-01 |
| Weights | Open | Open |
| Context window | — | 131K |
| Max output | — | 262K |
| Input $ / M tokens | — | $0.10 |
| Output $ / M tokens | — | $0.10 |
| Results tracked | 21 | 6 |
Sponsored placements are available on pages like this one. Advertise on Noometry
Category by category
Coding Not comparable
Llama 13b: 21.4 (#337), Ministral 3B: —
| Benchmark | Llama 13b | Ministral 3B |
|---|---|---|
| LMArena Coding | 683 | — |
Reasoning Ministral 3B leads
Llama 13b: 14.0 (#329), Ministral 3B: 18.4 (#282)
| Benchmark | Llama 13b | Ministral 3B |
|---|---|---|
| Epoch Capabilities Index | 100.58 | 118.1 |
| LMArena Hard Prompts | 728 | — |
| DTBench | — | 51.7% |
| LMCA | — | 5.5% |
| BIG-Bench Hard | 37.9% | — |
| HellaSwag | 79.2% | — |
| LAMBADA | 75.2% | — |
| PIQA | 80.1% | — |
| WinoGrande | 73% | — |
Math Too close to call
Llama 13b: 26.7 (#256), Ministral 3B: 26.6 (#258)
| Benchmark | Llama 13b | Ministral 3B |
|---|---|---|
| LMArena Math | 838 | — |
| MATH Level 5 | — | 14.4% |
| GSM8K | 20.6% | — |
Knowledge Not comparable
Llama 13b: —, Ministral 3B: 10.4 (#302)
| Benchmark | Llama 13b | Ministral 3B |
|---|---|---|
| GPQA Diamond | — | 25.3% |
| Vectara Hallucination Rate | — | 7.3% |
| ARC (AI2) Challenge | 52.7% | — |
| BoolQ | 78.7% | — |
| MMLU | 47.7% | — |
| OpenBookQA | 56.4% | — |
| TriviaQA | 77.9% | — |
Multimodal Not comparable
Llama 13b: —, Ministral 3B: —
| Benchmark | Llama 13b | Ministral 3B |
|---|---|---|
| ScienceQA | 43.3% | — |
Multilingual Not comparable
Llama 13b: 16.6 (#297), Ministral 3B: —
| Benchmark | Llama 13b | Ministral 3B |
|---|---|---|
| LMArena Non-English | 819 | — |
Instruction Following Not comparable
Llama 13b: 36.7 (#305), Ministral 3B: —
| Benchmark | Llama 13b | Ministral 3B |
|---|---|---|
| LMArena Instruction Following | 781 | — |
Writing & Preference Not comparable
Llama 13b: 13.8 (#312), Ministral 3B: —
| Benchmark | Llama 13b | Ministral 3B |
|---|---|---|
| LMArena Text | 834 | — |
| LMArena Creative Writing | 794 | — |
| LMArena Multi-Turn | 753 | — |
Frequently asked questions
Is Llama 13b better than Ministral 3B?
Ministral 3B is the stronger model overall, scoring 26.2 to 24.4 on the Noometry Index.
How many benchmarks do Llama 13b and Ministral 3B share?
1 benchmark has published results for both models. Llama 13b has 21 scored results on Noometry and Ministral 3B has 6.