Noometry

Model comparison

Mistral 7B vs Qwen Max

Qwen Max is the stronger model overall, scoring 34.7 to 23.0 on the Noometry Index. Mistral 7B costs 11× less per token, which makes it the better buy when Qwen Max's lead doesn't matter for your workload.

Last verified . 19 shared benchmarks.

Mistral 7B Mistral AI

23.0

Rank #351 Confirmed

Qwen Max Alibaba (Qwen)

34.7

Rank #230 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Mistral 7B scores higher in 0 categories and Qwen Max in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Qwen Max leads 30.3 to 7.4.
  • The biggest single-benchmark swing is MATH Level 5: 3.7% for Mistral 7B and 67.2% for Qwen Max.
  • Mistral 7B is cheaper at $0.25 / $0.25 per million input/output tokens, against $1.60 / $6.40 for Qwen Max.
  • Qwen Max accepts more context: 33K tokens versus 8K.
  • Mistral 7B has downloadable open weights; the other is API-only.

Side by side

Mistral 7BQwen Max
ProviderMistral AIAlibaba (Qwen)
Noometry Index23.034.7
Released2023-09-272024-04-03
WeightsOpenProprietary
Context window8K33K
Max output8K8K
Input $ / M tokens$0.25$1.60
Output $ / M tokens$0.25$6.40
Results tracked3723

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen Max leads

Mistral 7B: 26.4 (#326), Qwen Max: 30.7 (#292)

BenchmarkMistral 7BQwen Max
LMArena Coding10821288
Aider Polyglot—21.8%
BigCodeBench Instruct19.5%—
BigCodeBench Complete27.3%—
HumanEval+36%—
MBPP+42.1%—

Reasoning Qwen Max leads

Mistral 7B: 13.1 (#336), Qwen Max: 25.1 (#151)

BenchmarkMistral 7BQwen Max
LMArena Hard Prompts10671269
Chess Puzzles0%—
DTBench42.5%—
Adversarial NLI47.1%—
BIG-Bench Hard56.1%—
Epoch Capabilities Index112.21—
HellaSwag81%—
PIQA83%—
WinoGrande75.3%—

Math Qwen Max leads

Mistral 7B: 8.1 (#325), Qwen Max: 22.3 (#276)

BenchmarkMistral 7BQwen Max
OTIS Mock AIME 2024-20250.3%16.1%
LMArena Math10851275
MATH Level 53.7%67.2%
FrontierMath (Feb 2025 set)—1%
GSM8K54.4%—

Knowledge Qwen Max leads

Mistral 7B: 7.4 (#311), Qwen Max: 30.3 (#228)

BenchmarkMistral 7BQwen Max
GPQA Diamond15.2%56.1%
LMArena Expert10361248
ARC (AI2) Challenge78.6%—
BoolQ87.4%—
MMLU62.5%—
OpenBookQA79.8%—
TriviaQA75.2%—

Multilingual Qwen Max leads

Mistral 7B: 25.8 (#283), Qwen Max: 41.8 (#202)

BenchmarkMistral 7BQwen Max
LMArena Non-English10121263
LMArena Chinese10091254
LMArena French10371330
LMArena German9871254
LMArena Japanese8781205
LMArena Russian10181274
LMArena Spanish10261290
LMArena Korean—1142

Instruction Following Qwen Max leads

Mistral 7B: 54.2 (#280), Qwen Max: 66.5 (#208)

BenchmarkMistral 7BQwen Max
LMArena Instruction Following10601262

Long Context Qwen Max leads

Mistral 7B: 32.2 (#271), Qwen Max: 39.4 (#180)

BenchmarkMistral 7BQwen Max
LMArena Longer Query10601288
Fiction.LiveBench—66.7%

Writing & Preference Qwen Max leads

Mistral 7B: 30.7 (#286), Qwen Max: 47.8 (#205)

BenchmarkMistral 7BQwen Max
LMArena Text10901282
LMArena Creative Writing10681248
LMArena Multi-Turn10621277

Frequently asked questions

Is Mistral 7B better than Qwen Max?

Qwen Max is the stronger model overall, scoring 34.7 to 23.0 on the Noometry Index. Mistral 7B costs 11× less per token, which makes it the better buy when Qwen Max's lead doesn't matter for your workload.

Which is cheaper, Mistral 7B or Qwen Max?

Mistral 7B is cheaper. It lists at $0.25 per million input tokens and $0.25 per million output tokens; Qwen Max lists at $1.60 and $6.40.

Is Mistral 7B or Qwen Max better for coding?

Qwen Max scores higher on coding benchmarks: 30.7 versus 26.4 in the Noometry coding category.

Which has the bigger context window?

Qwen Max does, with 33K tokens against 8K.

How many benchmarks do Mistral 7B and Qwen Max share?

19 benchmarks have published results for both models. Mistral 7B has 37 scored results on Noometry and Qwen Max has 23.

Related comparisons

Go deeper