Model comparison

Llama 3.2 90B vs Mistral 7B

Llama 3.2 90B is the stronger model overall, scoring 27.5 to 23.0 on the Noometry Index.

Last verified . 5 shared benchmarks.

Llama 3.2 90B Meta

27.5

Rank #331 Confirmed

Mistral 7B Mistral AI

23.0

Rank #351 Confirmed

Summary

  • They share 5 benchmarks with published results for both. Llama 3.2 90B scores higher in 3 categories and Mistral 7B in 0 categories; 3 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Llama 3.2 90B leads 21.7 to 7.4.
  • The biggest single-benchmark swing is MATH Level 5: 39.4% for Llama 3.2 90B and 3.7% for Mistral 7B.

Side by side

Llama 3.2 90B and Mistral 7B specifications
Llama 3.2 90BMistral 7B
ProviderMetaMistral AI
Noometry Index27.523.0
Released2024-09-242023-09-27
WeightsOpenOpen
Context window—8K
Max output—8K
Input $ / M tokens—$0.25
Output $ / M tokens—$0.25
Results tracked937

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Llama 3.2 90B: —, Mistral 7B: 26.4 (#326)

Coding benchmarks
BenchmarkLlama 3.2 90BMistral 7B
BigCodeBench Instruct—19.5%
LMArena Coding—1082
BigCodeBench Complete—27.3%
HumanEval+—36%
MBPP+—42.1%

Agentic & Tool Use Not comparable

Llama 3.2 90B: 30.0 (#80), Mistral 7B: —

Agentic & Tool Use benchmarks
BenchmarkLlama 3.2 90BMistral 7B
BALROG27.3%—

Reasoning Llama 3.2 90B leads

Llama 3.2 90B: 21.7 (#217), Mistral 7B: 13.1 (#336)

Reasoning benchmarks
BenchmarkLlama 3.2 90BMistral 7B
Epoch Capabilities Index125.5112.21
Chess Puzzles—0%
EnigmaEval0.4%—
LMArena Hard Prompts—1067
DTBench—42.5%
Adversarial NLI—47.1%
BIG-Bench Hard—56.1%
HellaSwag—81%
PIQA—83%
WinoGrande—75.3%

Math Llama 3.2 90B leads

Llama 3.2 90B: 11.1 (#308), Mistral 7B: 8.1 (#325)

Math benchmarks
BenchmarkLlama 3.2 90BMistral 7B
OTIS Mock AIME 2024-20252.6%0.3%
MATH Level 539.4%3.7%
LMArena Math—1085
GSM8K—54.4%

Knowledge Llama 3.2 90B leads

Llama 3.2 90B: 21.7 (#274), Mistral 7B: 7.4 (#311)

Knowledge benchmarks
BenchmarkLlama 3.2 90BMistral 7B
GPQA Diamond41%15.2%
MMLU80.3%62.5%
LMArena Expert—1036
ARC (AI2) Challenge—78.6%
BoolQ—87.4%
OpenBookQA—79.8%
TriviaQA—75.2%

Multimodal Not comparable

Llama 3.2 90B: 25.4 (#124), Mistral 7B: —

Multimodal benchmarks
BenchmarkLlama 3.2 90BMistral 7B
LMArena Vision1000—
GeoBench52%—

Multilingual Not comparable

Llama 3.2 90B: —, Mistral 7B: 25.8 (#283)

Multilingual benchmarks
BenchmarkLlama 3.2 90BMistral 7B
LMArena Non-English—1012
LMArena Chinese—1009
LMArena French—1037
LMArena German—987
LMArena Japanese—878
LMArena Russian—1018
LMArena Spanish—1026

Instruction Following Not comparable

Llama 3.2 90B: —, Mistral 7B: 54.2 (#280)

Instruction Following benchmarks
BenchmarkLlama 3.2 90BMistral 7B
LMArena Instruction Following—1060

Long Context Not comparable

Llama 3.2 90B: —, Mistral 7B: 32.2 (#271)

Long Context benchmarks
BenchmarkLlama 3.2 90BMistral 7B
LMArena Longer Query—1060

Writing & Preference Not comparable

Llama 3.2 90B: —, Mistral 7B: 30.7 (#286)

Writing & Preference benchmarks
BenchmarkLlama 3.2 90BMistral 7B
LMArena Text—1090
LMArena Creative Writing—1068
LMArena Multi-Turn—1062

Frequently asked questions

Is Llama 3.2 90B better than Mistral 7B?

Llama 3.2 90B is the stronger model overall, scoring 27.5 to 23.0 on the Noometry Index.

How many benchmarks do Llama 3.2 90B and Mistral 7B share?

5 benchmarks have published results for both models. Llama 3.2 90B has 9 scored results on Noometry and Mistral 7B has 37.

Related comparisons

Go deeper