Noometry

Model comparison

Mixtral 8x22B vs Qwen2.5 14B Instruct

Mixtral 8x22B has enough public results to be ranked (#330); Qwen2.5 14B Instruct does not yet, so treat this comparison as directional.

Last verified . 3 shared benchmarks.

Mixtral 8x22B Mistral AI

26.7

Rank #330 Confirmed

Qwen2.5 14B Instruct Alibaba (Qwen)

38.2

Unranked Sparse

Summary

  • They share 3 benchmarks with published results for both. Mixtral 8x22B scores higher in 0 categories and Qwen2.5 14B Instruct in 1 category; one gap is clear of the uncertainty.
  • The widest gap is in coding, where Qwen2.5 14B Instruct leads 37.1 to 23.8.
  • Qwen2.5 14B Instruct is cheaper at $0.35 / $1.40 per million input/output tokens, against $2 / $6 for Mixtral 8x22B.
  • Qwen2.5 14B Instruct accepts more context: 131K tokens versus 64K.

Side by side

Mixtral 8x22BQwen2.5 14B Instruct
ProviderMistral AIAlibaba (Qwen)
Noometry Index26.738.2
Released2024-04-172024-09
WeightsOpenOpen
Context window64K131K
Max output64K8K
Input $ / M tokens$2$0.35
Output $ / M tokens$6$1.40
Results tracked343

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5 14B Instruct leads

Mixtral 8x22B: 23.8 (#324), Qwen2.5 14B Instruct: 37.1 (#187)

BenchmarkMixtral 8x22BQwen2.5 14B Instruct
BigCodeBench Instruct40.6%39.8%
BigCodeBench Complete50.2%52.2%
WeirdML3.2%—
LMArena Coding1166—
HumanEval+72%—
MBPP+64.3%—

Agentic & Tool Use Not comparable

Mixtral 8x22B: 22.8 (#119), Qwen2.5 14B Instruct: —

BenchmarkMixtral 8x22BQwen2.5 14B Instruct
Cybench7.5%—

Reasoning Not comparable

Mixtral 8x22B: 19.1 (#246), Qwen2.5 14B Instruct: —

BenchmarkMixtral 8x22BQwen2.5 14B Instruct
LMArena Hard Prompts1150—
DTBench55.1%—
Epoch Capabilities Index122—
ForecastBench56.3—

Math Not comparable

Mixtral 8x22B: 21.2 (#271), Qwen2.5 14B Instruct: —

BenchmarkMixtral 8x22BQwen2.5 14B Instruct
Omni-MATH16.3%—
LMArena Math1184—
MATH Level 524.2%—

Knowledge Not comparable

Mixtral 8x22B: 15.1 (#288), Qwen2.5 14B Instruct: —

BenchmarkMixtral 8x22BQwen2.5 14B Instruct
MMLU77.8%79.9%
GPQA Diamond34.1%—
MMLU-Pro46%—
GPQA (HELM)33.4%—
LMArena Expert1113—

Multilingual Not comparable

Mixtral 8x22B: 32.9 (#250), Qwen2.5 14B Instruct: —

BenchmarkMixtral 8x22BQwen2.5 14B Instruct
LMArena Non-English1128—
LMArena Chinese1116—
LMArena French1165—
LMArena German1141—
LMArena Japanese1037—
LMArena Korean1056—
LMArena Russian1158—
LMArena Spanish1150—

Instruction Following Not comparable

Mixtral 8x22B: 57.9 (#261), Qwen2.5 14B Instruct: —

BenchmarkMixtral 8x22BQwen2.5 14B Instruct
IFEval72.4%—
LMArena Instruction Following1147—

Long Context Not comparable

Mixtral 8x22B: 34.6 (#242), Qwen2.5 14B Instruct: —

BenchmarkMixtral 8x22BQwen2.5 14B Instruct
LMArena Longer Query1144—

Writing & Preference Not comparable

Mixtral 8x22B: 37.1 (#257), Qwen2.5 14B Instruct: —

BenchmarkMixtral 8x22BQwen2.5 14B Instruct
LMArena Text1162—
LMArena Creative Writing1141—
WildBench71.1%—
LMArena Multi-Turn1130—

Frequently asked questions

Is Mixtral 8x22B better than Qwen2.5 14B Instruct?

Mixtral 8x22B has enough public results to be ranked (#330); Qwen2.5 14B Instruct does not yet, so treat this comparison as directional.

Which is cheaper, Mixtral 8x22B or Qwen2.5 14B Instruct?

Qwen2.5 14B Instruct is cheaper. It lists at $0.35 per million input tokens and $1.40 per million output tokens; Mixtral 8x22B lists at $2 and $6.

Is Mixtral 8x22B or Qwen2.5 14B Instruct better for coding?

Qwen2.5 14B Instruct scores higher on coding benchmarks: 37.1 versus 23.8 in the Noometry coding category.

Which has the bigger context window?

Qwen2.5 14B Instruct does, with 131K tokens against 64K.

How many benchmarks do Mixtral 8x22B and Qwen2.5 14B Instruct share?

3 benchmarks have published results for both models. Mixtral 8x22B has 34 scored results on Noometry and Qwen2.5 14B Instruct has 3.

Related comparisons

Go deeper