Noometry

Model comparison

Mistral Medium vs Qwen1.5-72B

Mistral Medium is the stronger model overall, scoring 36.3 to 30.8 on the Noometry Index.

Last verified . 18 shared benchmarks.

Mistral Medium Mistral AI

36.3

Rank #218 Confirmed

Qwen1.5-72B Alibaba (Qwen)

30.8

Rank #285 Confirmed

Summary

  • They share 18 benchmarks with published results for both. Mistral Medium scores higher in 7 categories and Qwen1.5-72B in 1 category; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Mistral Medium leads 60.0 to 37.3.
  • The biggest single-benchmark swing is GPQA Diamond: 59.5% for Mistral Medium and 28.8% for Qwen1.5-72B.

Side by side

Mistral MediumQwen1.5-72B
ProviderMistral AIAlibaba (Qwen)
Noometry Index36.330.8
Released2023-12-112024-02-04
WeightsOpenOpen
Context window262K—
Max output262K—
Input $ / M tokens$1.50—
Output $ / M tokens$7.50—
Results tracked3622

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Medium leads

Mistral Medium: 34.2 (#243), Qwen1.5-72B: 31.9 (#277)

BenchmarkMistral MediumQwen1.5-72B
LMArena Coding14341165
FrontierCode8%—
SciCode40.2%—
WeirdML43.7%—
BigCodeBench Instruct—33.2%
BigCodeBench Complete—40.3%
ALE-Bench763.98—
HumanEval+—59.1%
MBPP+—61.6%

Agentic & Tool Use Not comparable

Mistral Medium: 28.3 (#90), Qwen1.5-72B: —

BenchmarkMistral MediumQwen1.5-72B
Berkeley Function Calling Leaderboard37.7%—

Reasoning Mistral Medium leads

Mistral Medium: 24.0 (#167), Qwen1.5-72B: 22.2 (#203)

BenchmarkMistral MediumQwen1.5-72B
LMArena Hard Prompts14261148
Kagi LLM Benchmark50%—
CritPt0%—
DTBench75.5%—
LMCA26.1%—
Surface Evolver Bench26.9%—

Math Qwen1.5-72B leads

Mistral Medium: 28.1 (#245), Qwen1.5-72B: 33.2 (#205)

BenchmarkMistral MediumQwen1.5-72B
LMArena Math14081164
OTIS Mock AIME 2024-202532.2%—
ProofBench9%—
MATH Level 581.6%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge Mistral Medium leads

Mistral Medium: 25.0 (#265), Qwen1.5-72B: 11.5 (#300)

BenchmarkMistral MediumQwen1.5-72B
GPQA Diamond59.5%28.8%
LMArena Expert14081136
Humanity's Last Exam4.5%—
Vectara Hallucination Rate22.7%—

Multimodal Not comparable

Mistral Medium: 35.3 (#88), Qwen1.5-72B: —

BenchmarkMistral MediumQwen1.5-72B
LMArena Vision1172—

Multilingual Mistral Medium leads

Mistral Medium: 52.1 (#91), Qwen1.5-72B: 33.2 (#253)

BenchmarkMistral MediumQwen1.5-72B
LMArena Non-English14081135
LMArena Chinese14471186
LMArena French14591159
LMArena German14321084
LMArena Japanese13781061
LMArena Korean13801050
LMArena Russian14111104
LMArena Spanish14331110

Instruction Following Mistral Medium leads

Mistral Medium: 73.7 (#116), Qwen1.5-72B: 59.3 (#256)

BenchmarkMistral MediumQwen1.5-72B
LMArena Instruction Following13981141

Long Context Mistral Medium leads

Mistral Medium: 42.9 (#114), Qwen1.5-72B: 35.1 (#243)

BenchmarkMistral MediumQwen1.5-72B
LMArena Longer Query14061157

Writing & Preference Mistral Medium leads

Mistral Medium: 60.0 (#103), Qwen1.5-72B: 37.3 (#258)

BenchmarkMistral MediumQwen1.5-72B
LMArena Text14241166
LMArena Creative Writing13911137
LMArena Multi-Turn14181160
Short-Story Creative Writing77.3%—

Frequently asked questions

Is Mistral Medium better than Qwen1.5-72B?

Mistral Medium is the stronger model overall, scoring 36.3 to 30.8 on the Noometry Index.

Is Mistral Medium or Qwen1.5-72B better for coding?

Mistral Medium scores higher on coding benchmarks: 34.2 versus 31.9 in the Noometry coding category.

How many benchmarks do Mistral Medium and Qwen1.5-72B share?

18 benchmarks have published results for both models. Mistral Medium has 36 scored results on Noometry and Qwen1.5-72B has 22.

Related comparisons

Go deeper