Noometry

Model comparison

Mistral Large vs Qwen2.5-Max

Qwen2.5-Max is the stronger model overall, scoring 40.7 to 31.9 on the Noometry Index.

Last verified . 27 shared benchmarks.

Mistral Large Mistral AI

31.9

Rank #263 Confirmed

Qwen2.5-Max Alibaba (Qwen)

40.7

Rank #146 Confirmed

Summary

  • They share 27 benchmarks with published results for both. Mistral Large scores higher in 0 categories and Qwen2.5-Max in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Qwen2.5-Max leads 36.9 to 18.2.
  • The biggest single-benchmark swing is LiveBench Data Analysis: 50.1% for Mistral Large and 67.9% for Qwen2.5-Max.
  • Mistral Large has downloadable open weights; the other is API-only.

Side by side

Mistral LargeQwen2.5-Max
ProviderMistral AIAlibaba (Qwen)
Noometry Index31.940.7
Released2024-02-262025-01-25
WeightsOpenProprietary
Context window131K—
Max output16K—
Input $ / M tokens$2—
Output $ / M tokens$6—
Results tracked5127

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5-Max leads

Mistral Large: 34.3 (#240), Qwen2.5-Max: 41.8 (#117)

BenchmarkMistral LargeQwen2.5-Max
LiveBench Coding47.1%64.4%
LMArena Coding12771359
SciCode36.2%—
BigCodeBench Instruct30%—
BigCodeBench Complete38.3%—
ALE-Bench264.7—
HumanEval+62.2%—
MBPP+59.5%—

Agentic & Tool Use Not comparable

Mistral Large: 28.6 (#89), Qwen2.5-Max: —

BenchmarkMistral LargeQwen2.5-Max
Berkeley Function Calling Leaderboard38.4%—

Reasoning Qwen2.5-Max leads

Mistral Large: 15.8 (#310), Qwen2.5-Max: 25.6 (#147)

BenchmarkMistral LargeQwen2.5-Max
LiveBench Reasoning43.5%51.4%
LMArena Hard Prompts12571360
LiveBench Data Analysis50.1%67.9%
Epoch Capabilities Index128.52132.53
LiveBench48.4%62.3%
SimpleBench22.5%—
CritPt0%—
DTBench65.1%—
LMCA16.7%—
ForecastBench57.1—

Math Qwen2.5-Max leads

Mistral Large: 18.2 (#291), Qwen2.5-Max: 36.9 (#162)

BenchmarkMistral LargeQwen2.5-Max
LiveBench Math42.5%58.4%
LMArena Math12621369
OTIS Mock AIME 2024-20258.5%—
Omni-MATH28.1%—
MATH Level 550.3%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge Qwen2.5-Max leads

Mistral Large: 30.1 (#230), Qwen2.5-Max: 35.3 (#186)

BenchmarkMistral LargeQwen2.5-Max
Confabulations21.4%21.8%
LMArena Expert12321337
GPQA Diamond51.3%—
MMLU-Pro59.9%—
Vectara Hallucination Rate4.5%—
GPQA (HELM)43.5%—
MMLU80%—

Multilingual Qwen2.5-Max leads

Mistral Large: 40.0 (#219), Qwen2.5-Max: 48.1 (#146)

BenchmarkMistral LargeQwen2.5-Max
LMArena Non-English12371352
LMArena Chinese12401382
LMArena French13251396
LMArena German12541350
LMArena Japanese11881300
LMArena Korean12021304
LMArena Russian12571353
LMArena Spanish12681377

Instruction Following Qwen2.5-Max leads

Mistral Large: 67.9 (#191), Qwen2.5-Max: 71.3 (#152)

BenchmarkMistral LargeQwen2.5-Max
LiveBench Instruction Following67.9%75.3%
LMArena Instruction Following12491335
IFEval87.7%—

Long Context Qwen2.5-Max leads

Mistral Large: 38.3 (#199), Qwen2.5-Max: 41.4 (#142)

BenchmarkMistral LargeQwen2.5-Max
LMArena Longer Query12611358

Writing & Preference Qwen2.5-Max leads

Mistral Large: 40.7 (#242), Qwen2.5-Max: 55.4 (#146)

BenchmarkMistral LargeQwen2.5-Max
LMArena Text12661367
LMArena Creative Writing12431339
Short-Story Creative Writing69%72.9%
LMArena Multi-Turn12601364
LiveBench Language39.4%56.3%
EQ-Bench Creative Writing985—
WildBench80.1%—

Frequently asked questions

Is Mistral Large better than Qwen2.5-Max?

Qwen2.5-Max is the stronger model overall, scoring 40.7 to 31.9 on the Noometry Index.

Is Mistral Large or Qwen2.5-Max better for coding?

Qwen2.5-Max scores higher on coding benchmarks: 41.8 versus 34.3 in the Noometry coding category.

How many benchmarks do Mistral Large and Qwen2.5-Max share?

27 benchmarks have published results for both models. Mistral Large has 51 scored results on Noometry and Qwen2.5-Max has 27.

Related comparisons

Go deeper