Noometry

Model comparison

Mixtral 8x7B vs Qwen Max

Qwen Max is the stronger model overall, scoring 34.7 to 27.1 on the Noometry Index. Mixtral 8x7B costs 4.0× less per token, which makes it the better buy when Qwen Max's lead doesn't matter for your workload.

Last verified . 19 shared benchmarks.

Mixtral 8x7B Mistral AI

27.1

Rank #334 Confirmed

Qwen Max Alibaba (Qwen)

34.7

Rank #230 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Mixtral 8x7B scores higher in 1 category and Qwen Max in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Qwen Max leads 30.3 to 11.0.
  • The biggest single-benchmark swing is MATH Level 5: 10% for Mixtral 8x7B and 67.2% for Qwen Max.
  • Mixtral 8x7B is cheaper at $0.70 / $0.70 per million input/output tokens, against $1.60 / $6.40 for Qwen Max.
  • Qwen Max accepts more context: 33K tokens versus 32K.
  • Mixtral 8x7B has downloadable open weights; the other is API-only.

Side by side

Mixtral 8x7BQwen Max
ProviderMistral AIAlibaba (Qwen)
Noometry Index27.134.7
Released2023-12-112024-04-03
WeightsOpenProprietary
Context window32K33K
Max output32K8K
Input $ / M tokens$0.70$1.60
Output $ / M tokens$0.70$6.40
Results tracked3823

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mixtral 8x7B leads

Mixtral 8x7B: 32.8 (#269), Qwen Max: 30.7 (#292)

BenchmarkMixtral 8x7BQwen Max
LMArena Coding11261288
Aider Polyglot—21.8%
HumanEval+39.6%—
MBPP+49.7%—

Reasoning Qwen Max leads

Mixtral 8x7B: 18.2 (#285), Qwen Max: 25.1 (#151)

BenchmarkMixtral 8x7BQwen Max
LMArena Hard Prompts11151269
DTBench49.6%—
Adversarial NLI55.2%—
Epoch Capabilities Index118.47—
ForecastBench56.3—
HellaSwag86.7%—
PIQA83.6%—
WinoGrande77.2%—

Math Qwen Max leads

Mixtral 8x7B: 18.8 (#289), Qwen Max: 22.3 (#276)

BenchmarkMixtral 8x7BQwen Max
LMArena Math11471275
MATH Level 510%67.2%
OTIS Mock AIME 2024-2025—16.1%
Omni-MATH10.5%—
FrontierMath (Feb 2025 set)—1%
GSM8K74.4%—

Knowledge Qwen Max leads

Mixtral 8x7B: 11.0 (#301), Qwen Max: 30.3 (#228)

BenchmarkMixtral 8x7BQwen Max
GPQA Diamond30.6%56.1%
LMArena Expert10881248
MMLU-Pro33.5%—
GPQA (HELM)29.6%—
ARC (AI2) Challenge87.3%—
MMLU70.6%—
OpenBookQA85.8%—
TriviaQA82.2%—

Multilingual Qwen Max leads

Mixtral 8x7B: 29.6 (#266), Qwen Max: 41.8 (#202)

BenchmarkMixtral 8x7BQwen Max
LMArena Non-English10771263
LMArena Chinese10551254
LMArena French11661330
LMArena German11141254
LMArena Japanese9311205
LMArena Korean9681142
LMArena Russian10901274
LMArena Spanish11111290

Instruction Following Qwen Max leads

Mixtral 8x7B: 51.0 (#297), Qwen Max: 66.5 (#208)

BenchmarkMixtral 8x7BQwen Max
LMArena Instruction Following11091262
IFEval57.5%—

Long Context Qwen Max leads

Mixtral 8x7B: 33.4 (#260), Qwen Max: 39.4 (#180)

BenchmarkMixtral 8x7BQwen Max
LMArena Longer Query11031288
Fiction.LiveBench—66.7%

Writing & Preference Qwen Max leads

Mixtral 8x7B: 34.2 (#270), Qwen Max: 47.8 (#205)

BenchmarkMixtral 8x7BQwen Max
LMArena Text11321282
LMArena Creative Writing11091248
LMArena Multi-Turn11151277
WildBench67.3%—

Frequently asked questions

Is Mixtral 8x7B better than Qwen Max?

Qwen Max is the stronger model overall, scoring 34.7 to 27.1 on the Noometry Index. Mixtral 8x7B costs 4.0× less per token, which makes it the better buy when Qwen Max's lead doesn't matter for your workload.

Which is cheaper, Mixtral 8x7B or Qwen Max?

Mixtral 8x7B is cheaper. It lists at $0.70 per million input tokens and $0.70 per million output tokens; Qwen Max lists at $1.60 and $6.40.

Is Mixtral 8x7B or Qwen Max better for coding?

Mixtral 8x7B scores higher on coding benchmarks: 32.8 versus 30.7 in the Noometry coding category.

Which has the bigger context window?

Qwen Max does, with 33K tokens against 32K.

How many benchmarks do Mixtral 8x7B and Qwen Max share?

19 benchmarks have published results for both models. Mixtral 8x7B has 38 scored results on Noometry and Qwen Max has 23.

Related comparisons

Go deeper