Model comparison

Llama 3.2 11B vs Mixtral 8x7B

Mixtral 8x7B has enough public results to be ranked (#334); Llama 3.2 11B does not yet, so treat this comparison as directional.

Last verified . 1 shared benchmarks.

Llama 3.2 11B Meta

32.1

Unranked Sparse

Mixtral 8x7B Mistral AI

27.1

Rank #334 Confirmed

Summary

  • They share 1 benchmark with published results for both.

Side by side

Llama 3.2 11B and Mixtral 8x7B specifications
Llama 3.2 11BMixtral 8x7B
ProviderMetaMistral AI
Noometry Index32.127.1
Released2024-09-242023-12-11
WeightsOpenOpen
Context window—32K
Max output—32K
Input $ / M tokens—$0.70
Output $ / M tokens—$0.70
Results tracked338

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Llama 3.2 11B: —, Mixtral 8x7B: 32.8 (#269)

Coding benchmarks
BenchmarkLlama 3.2 11BMixtral 8x7B
LMArena Coding—1126
HumanEval+—39.6%
MBPP+—49.7%

Agentic & Tool Use Not comparable

Llama 3.2 11B: 27.3, Mixtral 8x7B: —

Agentic & Tool Use benchmarks
BenchmarkLlama 3.2 11BMixtral 8x7B
BALROG16.8%—

Reasoning Not comparable

Llama 3.2 11B: —, Mixtral 8x7B: 18.2 (#285)

Reasoning benchmarks
BenchmarkLlama 3.2 11BMixtral 8x7B
LMArena Hard Prompts—1115
DTBench—49.6%
Adversarial NLI—55.2%
Epoch Capabilities Index—118.47
ForecastBench—56.3
HellaSwag—86.7%
PIQA—83.6%
WinoGrande—77.2%

Math Not comparable

Llama 3.2 11B: —, Mixtral 8x7B: 18.8 (#289)

Math benchmarks
BenchmarkLlama 3.2 11BMixtral 8x7B
Omni-MATH—10.5%
LMArena Math—1147
MATH Level 5—10%
GSM8K—74.4%

Knowledge Not comparable

Llama 3.2 11B: —, Mixtral 8x7B: 11.0 (#301)

Knowledge benchmarks
BenchmarkLlama 3.2 11BMixtral 8x7B
MMLU56.5%70.6%
GPQA Diamond—30.6%
MMLU-Pro—33.5%
GPQA (HELM)—29.6%
LMArena Expert—1088
ARC (AI2) Challenge—87.3%
OpenBookQA—85.8%
TriviaQA—82.2%

Multimodal Not comparable

Llama 3.2 11B: 24.4, Mixtral 8x7B: —

Multimodal benchmarks
BenchmarkLlama 3.2 11BMixtral 8x7B
LMArena Vision966—

Multilingual Not comparable

Llama 3.2 11B: —, Mixtral 8x7B: 29.6 (#266)

Multilingual benchmarks
BenchmarkLlama 3.2 11BMixtral 8x7B
LMArena Non-English—1077
LMArena Chinese—1055
LMArena French—1166
LMArena German—1114
LMArena Japanese—931
LMArena Korean—968
LMArena Russian—1090
LMArena Spanish—1111

Instruction Following Not comparable

Llama 3.2 11B: —, Mixtral 8x7B: 51.0 (#297)

Instruction Following benchmarks
BenchmarkLlama 3.2 11BMixtral 8x7B
IFEval—57.5%
LMArena Instruction Following—1109

Long Context Not comparable

Llama 3.2 11B: —, Mixtral 8x7B: 33.4 (#260)

Long Context benchmarks
BenchmarkLlama 3.2 11BMixtral 8x7B
LMArena Longer Query—1103

Writing & Preference Not comparable

Llama 3.2 11B: —, Mixtral 8x7B: 34.2 (#270)

Writing & Preference benchmarks
BenchmarkLlama 3.2 11BMixtral 8x7B
LMArena Text—1132
LMArena Creative Writing—1109
WildBench—67.3%
LMArena Multi-Turn—1115

Frequently asked questions

Is Llama 3.2 11B better than Mixtral 8x7B?

Mixtral 8x7B has enough public results to be ranked (#334); Llama 3.2 11B does not yet, so treat this comparison as directional.

How many benchmarks do Llama 3.2 11B and Mixtral 8x7B share?

1 benchmark has published results for both models. Llama 3.2 11B has 3 scored results on Noometry and Mixtral 8x7B has 38.

Related comparisons

Go deeper