Noometry

Model comparison

Mistral Small vs Qwen2.5-Max

Qwen2.5-Max is the stronger model overall, scoring 40.7 to 33.4 on the Noometry Index.

Last verified . 24 shared benchmarks.

Mistral Small Mistral AI

33.4

Rank #243 Confirmed

Qwen2.5-Max Alibaba (Qwen)

40.7

Rank #146 Confirmed

Summary

  • They share 24 benchmarks with published results for both. Mistral Small scores higher in 0 categories and Qwen2.5-Max in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Qwen2.5-Max leads 36.9 to 16.4.
  • The biggest single-benchmark swing is LiveBench Coding: 36.2% for Mistral Small and 64.4% for Qwen2.5-Max.
  • Mistral Small has downloadable open weights; the other is API-only.

Side by side

Mistral SmallQwen2.5-Max
ProviderMistral AIAlibaba (Qwen)
Noometry Index33.440.7
Released2024-02-262025-01-25
WeightsOpenProprietary
Context window262K—
Max output256K—
Input $ / M tokens$0.15—
Output $ / M tokens$0.60—
Results tracked3927

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5-Max leads

Mistral Small: 34.0 (#247), Qwen2.5-Max: 41.8 (#117)

BenchmarkMistral SmallQwen2.5-Max
LiveBench Coding36.2%64.4%
LMArena Coding13621359
SciCode26.5%—
BigCodeBench Instruct36.1%—
BigCodeBench Complete46.6%—
ALE-Bench497.62—

Agentic & Tool Use Not comparable

Mistral Small: 28.1 (#93), Qwen2.5-Max: —

BenchmarkMistral SmallQwen2.5-Max
Berkeley Function Calling Leaderboard37.1%—

Reasoning Qwen2.5-Max leads

Mistral Small: 19.8 (#250), Qwen2.5-Max: 25.6 (#147)

BenchmarkMistral SmallQwen2.5-Max
LiveBench Reasoning44.8%51.4%
LMArena Hard Prompts13351360
LiveBench Data Analysis53.7%67.9%
LiveBench44%62.3%
Kagi LLM Benchmark37.8%—
CritPt0%—
DTBench70.9%—
LMCA20.6%—
Epoch Capabilities Index—132.53

Math Qwen2.5-Max leads

Mistral Small: 16.4 (#293), Qwen2.5-Max: 36.9 (#162)

BenchmarkMistral SmallQwen2.5-Max
LiveBench Math39.9%58.4%
LMArena Math13411369
OTIS Mock AIME 2024-20255.8%—
MATH Level 546.8%—

Knowledge Qwen2.5-Max leads

Mistral Small: 31.0 (#222), Qwen2.5-Max: 35.3 (#186)

BenchmarkMistral SmallQwen2.5-Max
LMArena Expert12911337
GPQA Diamond47.5%—
Confabulations—21.8%
Vectara Hallucination Rate5.1%—
MMLU68.7%—

Multimodal Not comparable

Mistral Small: 33.5 (#96), Qwen2.5-Max: —

BenchmarkMistral SmallQwen2.5-Max
LMArena Vision1142—

Multilingual Qwen2.5-Max leads

Mistral Small: 45.5 (#169), Qwen2.5-Max: 48.1 (#146)

BenchmarkMistral SmallQwen2.5-Max
LMArena Non-English13151352
LMArena Chinese13401382
LMArena French13371396
LMArena German13401350
LMArena Japanese12751300
LMArena Korean12591304
LMArena Russian13241353
LMArena Spanish13461377

Instruction Following Qwen2.5-Max leads

Mistral Small: 66.4 (#209), Qwen2.5-Max: 71.3 (#152)

BenchmarkMistral SmallQwen2.5-Max
LiveBench Instruction Following63.7%75.3%
LMArena Instruction Following13101335

Long Context Qwen2.5-Max leads

Mistral Small: 40.4 (#156), Qwen2.5-Max: 41.4 (#142)

BenchmarkMistral SmallQwen2.5-Max
LMArena Longer Query13271358

Writing & Preference Qwen2.5-Max leads

Mistral Small: 52.5 (#171), Qwen2.5-Max: 55.4 (#146)

BenchmarkMistral SmallQwen2.5-Max
LMArena Text13381367
LMArena Creative Writing13051339
LMArena Multi-Turn13441364
LiveBench Language30.5%56.3%
Short-Story Creative Writing—72.9%

Frequently asked questions

Is Mistral Small better than Qwen2.5-Max?

Qwen2.5-Max is the stronger model overall, scoring 40.7 to 33.4 on the Noometry Index.

Is Mistral Small or Qwen2.5-Max better for coding?

Qwen2.5-Max scores higher on coding benchmarks: 41.8 versus 34.0 in the Noometry coding category.

How many benchmarks do Mistral Small and Qwen2.5-Max share?

24 benchmarks have published results for both models. Mistral Small has 39 scored results on Noometry and Qwen2.5-Max has 27.

Related comparisons

Go deeper