Model comparison

Devstral Small 2505 vs Gemini 2.5 Pro

Gemini 2.5 Pro is the stronger model overall, scoring 45.0 to 34.3 on the Noometry Index. Devstral Small 2505 costs 23× less per token, which makes it the better buy when Gemini 2.5 Pro's lead doesn't matter for your workload.

Last verified October 11, 2026. 4 shared benchmarks.

Devstral Small 2505

Mistral AI

34.3

Rank #233 Reported

Gemini 2.5 Pro

Google

45.0

Rank #75 Confirmed

Summary

They share 4 benchmarks with published results for both. Devstral Small 2505 scores higher in 0 categories and Gemini 2.5 Pro in 2 categories; 2 gaps are clear of the uncertainty.
The widest gap is in reasoning, where Gemini 2.5 Pro leads 28.8 to 19.7.
The biggest single-benchmark swing is Kagi LLM Benchmark: 37.7% for Devstral Small 2505 and 70.3% for Gemini 2.5 Pro.
Devstral Small 2505 is cheaper at $0.10 / $0.30 per million input/output tokens, against $1.25 / $10 for Gemini 2.5 Pro.
Gemini 2.5 Pro accepts more context: 1.05M tokens versus 128K.
Devstral Small 2505 has downloadable open weights; the other is API-only.

Side by side

Devstral Small 2505 and Gemini 2.5 Pro specifications
	Devstral Small 2505	Gemini 2.5 Pro
Provider	Mistral AI	Google
Noometry Index	34.3	45.0
Released	2025-05-07	2025-03-25
Weights	Open	Proprietary
Context window	128K	1.05M
Max output	128K	66K
Input $ / M tokens	$0.10	$1.25
Output $ / M tokens	$0.30	$10
Results tracked	4	78

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 2.5 Pro leads

Devstral Small 2505: 38.9 (#166), Gemini 2.5 Pro: 42.4 (#101)

Coding benchmarks
Benchmark	Devstral Small 2505	Gemini 2.5 Pro
SWE-bench Verified (bash only)	56.4%	53.6%
SciCode	28.8%	42.8%
SWE-bench Verified	—	57.6%
Aider Polyglot	—	83.1%
LMArena WebDev	—	1227
GSO	—	3.9%
WeirdML	—	54%
LiveBench Coding	—	85.9%
LMArena Coding	—	1452
CadEval	—	64%
ALE-Bench	—	785.52
AlgoTune	—	1.51

Agentic & Tool Use Not comparable

Devstral Small 2505: —, Gemini 2.5 Pro: 29.2 (#88)

Agentic & Tool Use benchmarks
Benchmark	Devstral Small 2505	Gemini 2.5 Pro
Terminal-Bench	—	32.6%
GDPval	—	23.3%
Remote Labor Index	—	0.8%
TheAgentCompany	—	30.3%
τ²-bench Banking	—	13.7%
DeepResearch Bench	—	42.8%
BALROG	—	43.3%
LMArena Search	—	1142
METR Time Horizons	—	55.4%
Vending-Bench 2	—	573.64

Reasoning Gemini 2.5 Pro leads

Devstral Small 2505: 19.7 (#252), Gemini 2.5 Pro: 28.8 (#99)

Reasoning benchmarks
Benchmark	Devstral Small 2505	Gemini 2.5 Pro
Kagi LLM Benchmark	37.7%	70.3%
CritPt	0%	2%
ARC-AGI-2	—	4.9%
SimpleBench	—	62.4%
ARC-AGI-1	—	41%
Chess Puzzles	—	20%
EnigmaEval	—	5.6%
LiveBench Reasoning	—	89.8%
LMArena Hard Prompts	—	1455
DTBench	—	82.4%
LiveBench Data Analysis	—	79.9%
LMCA	—	34.8%
Epoch Capabilities Index	—	145.32
ForecastBench	—	61.3
LiveBench	—	82.3%

Math Not comparable

Devstral Small 2505: —, Gemini 2.5 Pro: 32.5 (#213)

Math benchmarks
Benchmark	Devstral Small 2505	Gemini 2.5 Pro
FrontierMath (Tiers 1-3)	—	24.6%
FrontierMath Tier 4	—	0%
OTIS Mock AIME 2024-2025	—	84.7%
Omni-MATH	—	41.6%
LiveBench Math	—	90.2%
LMArena Math	—	1450
MATH Level 5	—	95.9%
FrontierMath (Feb 2025 set)	—	14.1%
FrontierMath Tier 4 (v1)	—	4.2%

Knowledge Not comparable

Devstral Small 2505: —, Gemini 2.5 Pro: 56.0 (#46)

Knowledge benchmarks
Benchmark	Devstral Small 2505	Gemini 2.5 Pro
GPQA Diamond	—	85.3%
Humanity's Last Exam	—	21.6%
MMLU-Pro	—	86.3%
Confabulations	—	10.6%
Vectara Hallucination Rate	—	7%
GPQA (HELM)	—	74.9%
LMArena Expert	—	1452

Multimodal Not comparable

Devstral Small 2505: —, Gemini 2.5 Pro: 45.2 (#18)

Multimodal benchmarks
Benchmark	Devstral Small 2505	Gemini 2.5 Pro
LMArena Vision	—	1263
GeoBench	—	86%
VPCT	—	48%
LMArena Document	—	1421
SpatialViz-Bench	—	44.7%

Multilingual Not comparable

Devstral Small 2505: —, Gemini 2.5 Pro: 55.3 (#31)

Multilingual benchmarks
Benchmark	Devstral Small 2505	Gemini 2.5 Pro
LMArena Non-English	—	1451
LMArena Chinese	—	1507
LMArena French	—	1472
LMArena German	—	1487
LMArena Japanese	—	1461
LMArena Korean	—	1434
LMArena Russian	—	1461
LMArena Spanish	—	1473

Instruction Following Not comparable

Devstral Small 2505: —, Gemini 2.5 Pro: 75.0 (#75)

Instruction Following benchmarks
Benchmark	Devstral Small 2505	Gemini 2.5 Pro
LiveBench Instruction Following	—	80.6%
IFEval	—	84%
LMArena Instruction Following	—	1437

Long Context Not comparable

Devstral Small 2505: —, Gemini 2.5 Pro: 59.8 (#5)

Long Context benchmarks
Benchmark	Devstral Small 2505	Gemini 2.5 Pro
Fiction.LiveBench	—	91.7%
LMArena Longer Query	—	1449

Writing & Preference Not comparable

Devstral Small 2505: —, Gemini 2.5 Pro: 63.7 (#62)

Writing & Preference benchmarks
Benchmark	Devstral Small 2505	Gemini 2.5 Pro
LMArena Text	—	1458
LMArena Creative Writing	—	1454
Short-Story Creative Writing	—	83.8%
EQ-Bench Creative Writing	—	1421
WildBench	—	85.7%
LMArena Multi-Turn	—	1453
LiveBench Language	—	67.8%

Frequently asked questions

Is Devstral Small 2505 better than Gemini 2.5 Pro?

Which is cheaper, Devstral Small 2505 or Gemini 2.5 Pro?

Devstral Small 2505 is cheaper. It lists at $0.10 per million input tokens and $0.30 per million output tokens; Gemini 2.5 Pro lists at $1.25 and $10.

Is Devstral Small 2505 or Gemini 2.5 Pro better for coding?

Gemini 2.5 Pro scores higher on coding benchmarks: 42.4 versus 38.9 in the Noometry coding category.

Which has the bigger context window?

Gemini 2.5 Pro does, with 1.05M tokens against 128K.

How many benchmarks do Devstral Small 2505 and Gemini 2.5 Pro share?

4 benchmarks have published results for both models. Devstral Small 2505 has 4 scored results on Noometry and Gemini 2.5 Pro has 78.