Noometry

Model comparison

DeepSeek LLM 67B vs Mixtral 8x22B

Mixtral 8x22B is the stronger model overall, scoring 27.1 to 24.9 on the Noometry Index.

Last verified . 13 shared benchmarks.

DeepSeek LLM 67B DeepSeek

24.9

Rank #347 Confirmed

Mixtral 8x22B Mistral AI

27.1

Rank #333 Confirmed

Summary

  • They share 13 benchmarks with published results for both. DeepSeek LLM 67B scores higher in 1 category and Mixtral 8x22B in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Mixtral 8x22B leads 22.9 to 8.7.
  • The biggest single-benchmark swing is MATH Level 5: 6.4% for DeepSeek LLM 67B and 24.2% for Mixtral 8x22B.

Side by side

DeepSeek LLM 67BMixtral 8x22B
ProviderDeepSeekMistral AI
Noometry Index24.927.1
Released2023-11-292024-04-17
WeightsOpenOpen
Context window—64K
Max output—64K
Input $ / M tokens—$2
Output $ / M tokens—$6
Results tracked1534

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding DeepSeek LLM 67B leads

DeepSeek LLM 67B: 31.9 (#278), Mixtral 8x22B: 24.2 (#329)

BenchmarkDeepSeek LLM 67BMixtral 8x22B
LMArena Coding10961166
WeirdML—3.2%
BigCodeBench Instruct—40.6%
BigCodeBench Complete—50.2%
HumanEval+—72%
MBPP+—64.3%

Agentic & Tool Use Not comparable

DeepSeek LLM 67B: —, Mixtral 8x22B: 23.1 (#127)

BenchmarkDeepSeek LLM 67BMixtral 8x22B
Cybench—7.5%

Reasoning Mixtral 8x22B leads

DeepSeek LLM 67B: 16.5 (#304), Mixtral 8x22B: 19.9 (#248)

BenchmarkDeepSeek LLM 67BMixtral 8x22B
LMArena Hard Prompts10701150
Epoch Capabilities Index110.5122.03
Chess Puzzles0%—
DTBench—55.1%
ForecastBench—56.3

Math Mixtral 8x22B leads

DeepSeek LLM 67B: 8.7 (#324), Mixtral 8x22B: 22.9 (#275)

BenchmarkDeepSeek LLM 67BMixtral 8x22B
LMArena Math11081184
MATH Level 56.4%24.2%
OTIS Mock AIME 2024-20250.8%—
Omni-MATH—16.3%

Knowledge Mixtral 8x22B leads

DeepSeek LLM 67B: 7.0 (#313), Mixtral 8x22B: 15.1 (#293)

BenchmarkDeepSeek LLM 67BMixtral 8x22B
GPQA Diamond24.6%34.1%
MMLU-Pro—46%
GPQA (HELM)—33.4%
LMArena Expert—1113
MMLU—77.8%

Multilingual Mixtral 8x22B leads

DeepSeek LLM 67B: 29.4 (#267), Mixtral 8x22B: 32.8 (#255)

BenchmarkDeepSeek LLM 67BMixtral 8x22B
LMArena Non-English10731128
LMArena Chinese11321116
LMArena French—1166
LMArena German—1141
LMArena Japanese—1037
LMArena Korean—1057
LMArena Russian—1158
LMArena Spanish—1151

Instruction Following Mixtral 8x22B leads

DeepSeek LLM 67B: 55.4 (#277), Mixtral 8x22B: 57.7 (#266)

BenchmarkDeepSeek LLM 67BMixtral 8x22B
LMArena Instruction Following10791147
IFEval—72.4%

Long Context Mixtral 8x22B leads

DeepSeek LLM 67B: 33.1 (#265), Mixtral 8x22B: 34.7 (#247)

BenchmarkDeepSeek LLM 67BMixtral 8x22B
LMArena Longer Query10921144

Writing & Preference Mixtral 8x22B leads

DeepSeek LLM 67B: 31.6 (#282), Mixtral 8x22B: 36.9 (#262)

BenchmarkDeepSeek LLM 67BMixtral 8x22B
LMArena Text11051162
LMArena Creative Writing10671141
LMArena Multi-Turn10821130
WildBench—71.1%

Frequently asked questions

Is DeepSeek LLM 67B better than Mixtral 8x22B?

Mixtral 8x22B is the stronger model overall, scoring 27.1 to 24.9 on the Noometry Index.

Is DeepSeek LLM 67B or Mixtral 8x22B better for coding?

DeepSeek LLM 67B scores higher on coding benchmarks: 31.9 versus 24.2 in the Noometry coding category.

How many benchmarks do DeepSeek LLM 67B and Mixtral 8x22B share?

13 benchmarks have published results for both models. DeepSeek LLM 67B has 15 scored results on Noometry and Mixtral 8x22B has 34.

Related comparisons

Go deeper