Model comparison

Devstral Small 2505 vs Gemma 4 26B A4B IT

Gemma 4 26B A4B IT is the stronger model overall, scoring 43.5 to 34.3 on the Noometry Index.

Last verified October 10, 2026. 2 shared benchmarks.

Devstral Small 2505

Mistral AI

34.3

Rank #233 Reported

Gemma 4 26B A4B IT

Google

43.5

Rank #92 Confirmed

Summary

They share 2 benchmarks with published results for both. Devstral Small 2505 scores higher in 0 categories and Gemma 4 26B A4B IT in 2 categories; one gap is clear of the uncertainty.
The biggest single-benchmark swing is SciCode: 28.8% for Devstral Small 2505 and 40% for Gemma 4 26B A4B IT.
Gemma 4 26B A4B IT is cheaper at $0.0675 / $0.23 per million input/output tokens, against $0.10 / $0.30 for Devstral Small 2505.
Gemma 4 26B A4B IT accepts more context: 262K tokens versus 128K.

Side by side

Devstral Small 2505 and Gemma 4 26B A4B IT specifications
	Devstral Small 2505	Gemma 4 26B A4B IT
Provider	Mistral AI	Google
Noometry Index	34.3	43.5
Released	2025-05-07	2026-04-02
Weights	Open	Open
Context window	128K	262K
Max output	128K	33K
Input $ / M tokens	$0.10	$0.0675
Output $ / M tokens	$0.30	$0.23
Results tracked	4	28

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

Devstral Small 2505: 38.9 (#166), Gemma 4 26B A4B IT: 39.0 (#164)

Coding benchmarks
Benchmark	Devstral Small 2505	Gemma 4 26B A4B IT
SciCode	28.8%	40%
SWE-bench Verified (bash only)	56.4%	—
LMArena WebDev	—	1359
WeirdML	—	35.2%
LMArena Coding	—	1447
ALE-Bench	—	927.17

Reasoning Gemma 4 26B A4B IT leads

Devstral Small 2505: 19.7 (#252), Gemma 4 26B A4B IT: 21.8 (#213)

Reasoning benchmarks
Benchmark	Devstral Small 2505	Gemma 4 26B A4B IT
CritPt	0%	0%
Kagi LLM Benchmark	37.7%	—
Chess Puzzles	—	6%
LMArena Hard Prompts	—	1439
DTBench	—	74.9%
LMCA	—	29.7%
Epoch Capabilities Index	—	141.85

Math Not comparable

Devstral Small 2505: —, Gemma 4 26B A4B IT: 47.6 (#67)

Math benchmarks
Benchmark	Devstral Small 2505	Gemma 4 26B A4B IT
OTIS Mock AIME 2024-2025	—	82.2%
LMArena Math	—	1470

Knowledge Not comparable

Devstral Small 2505: —, Gemma 4 26B A4B IT: 45.7 (#85)

Knowledge benchmarks
Benchmark	Devstral Small 2505	Gemma 4 26B A4B IT
GPQA Diamond	—	73.2%
Vectara Hallucination Rate	—	5.2%
LMArena Expert	—	1447

Multimodal Not comparable

Devstral Small 2505: —, Gemma 4 26B A4B IT: 40.6 (#46)

Multimodal benchmarks
Benchmark	Devstral Small 2505	Gemma 4 26B A4B IT
LMArena Vision	—	1260

Multilingual Not comparable

Devstral Small 2505: —, Gemma 4 26B A4B IT: 53.1 (#71)

Multilingual benchmarks
Benchmark	Devstral Small 2505	Gemma 4 26B A4B IT
LMArena Non-English	—	1421
LMArena Chinese	—	1495
LMArena French	—	1460
LMArena Russian	—	1434
LMArena Spanish	—	1417

Instruction Following Not comparable

Devstral Small 2505: —, Gemma 4 26B A4B IT: 74.8 (#82)

Instruction Following benchmarks
Benchmark	Devstral Small 2505	Gemma 4 26B A4B IT
LMArena Instruction Following	—	1420

Long Context Not comparable

Devstral Small 2505: —, Gemma 4 26B A4B IT: 43.6 (#91)

Long Context benchmarks
Benchmark	Devstral Small 2505	Gemma 4 26B A4B IT
LMArena Longer Query	—	1428

Writing & Preference Not comparable

Devstral Small 2505: —, Gemma 4 26B A4B IT: 58.6 (#115)

Writing & Preference benchmarks
Benchmark	Devstral Small 2505	Gemma 4 26B A4B IT
LMArena Text	—	1434
LMArena Creative Writing	—	1402
EQ-Bench Creative Writing	—	1305
LMArena Multi-Turn	—	1441

Frequently asked questions

Is Devstral Small 2505 better than Gemma 4 26B A4B IT?

Gemma 4 26B A4B IT is the stronger model overall, scoring 43.5 to 34.3 on the Noometry Index.

Which is cheaper, Devstral Small 2505 or Gemma 4 26B A4B IT?

Gemma 4 26B A4B IT is cheaper. It lists at $0.0675 per million input tokens and $0.23 per million output tokens; Devstral Small 2505 lists at $0.10 and $0.30.

Is Devstral Small 2505 or Gemma 4 26B A4B IT better for coding?

They score almost the same on coding (38.9 vs 39.0); test both on your own repository before choosing.

Which has the bigger context window?

Gemma 4 26B A4B IT does, with 262K tokens against 128K.

How many benchmarks do Devstral Small 2505 and Gemma 4 26B A4B IT share?

2 benchmarks have published results for both models. Devstral Small 2505 has 4 scored results on Noometry and Gemma 4 26B A4B IT has 28.