Model comparison

Devstral Small 2505 vs GPT-6.1 Sol

GPT-6.1 Sol is the stronger model overall, scoring 65.6 to 34.3 on the Noometry Index. Devstral Small 2505 costs 27× less per token, which makes it the better buy when GPT-6.1 Sol's lead doesn't matter for your workload.

Last verified October 11, 2026. 2 shared benchmarks.

Devstral Small 2505

Mistral AI

34.3

Rank #233 Reported

GPT-6.1 Sol OpenAI

65.6

Rank #6 Confirmed

Summary

They share 2 benchmarks with published results for both. Devstral Small 2505 scores higher in 0 categories and GPT-6.1 Sol in 2 categories; 2 gaps are clear of the uncertainty.
The widest gap is in reasoning, where GPT-6.1 Sol leads 81.9 to 19.7.
The biggest single-benchmark swing is CritPt: 0% for Devstral Small 2505 and 31.7% for GPT-6.1 Sol.
Devstral Small 2505 is cheaper at $0.10 / $0.30 per million input/output tokens, against $2 / $10 for GPT-6.1 Sol.
GPT-6.1 Sol accepts more context: 1.05M tokens versus 128K.
Devstral Small 2505 has downloadable open weights; the other is API-only.

Side by side

Devstral Small 2505 and GPT-6.1 Sol specifications
	Devstral Small 2505	GPT-6.1 Sol
Provider	Mistral AI	OpenAI
Noometry Index	34.3	65.6
Released	2025-05-07	2026-09-29
Weights	Open	Proprietary
Context window	128K	1.05M
Max output	128K	128K
Input $ / M tokens	$0.10	$2
Output $ / M tokens	$0.30	$10
Results tracked	4	34

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-6.1 Sol leads

Devstral Small 2505: 38.9 (#166), GPT-6.1 Sol: 63.2 (#8)

Coding benchmarks
Benchmark	Devstral Small 2505	GPT-6.1 Sol
SciCode	28.8%	55.8%
DeepSWE	—	75.2%
FrontierCode	—	50.2%
SWE-bench Verified (bash only)	56.4%	—
LMArena WebDev	—	1755
LMArena Coding	—	1487

Agentic & Tool Use Not comparable

Devstral Small 2505: —, GPT-6.1 Sol: 39.6 (#26)

Agentic & Tool Use benchmarks
Benchmark	Devstral Small 2505	GPT-6.1 Sol
APEX-Agents	—	60%
GDP.pdf	—	32%

Reasoning GPT-6.1 Sol leads

Devstral Small 2505: 19.7 (#252), GPT-6.1 Sol: 81.9 (#2)

Reasoning benchmarks
Benchmark	Devstral Small 2505	GPT-6.1 Sol
CritPt	0%	31.7%
ARC-AGI-2	—	94.2%
Kagi LLM Benchmark	37.7%	—
NYT Connections (extended)	—	95.5%
ARC-AGI-1	—	98.5%
Chess Puzzles	—	61%
EBR-Bench	—	54.3%
LMArena Hard Prompts	—	1466
Mystery Game Puzzles	—	80%
Epoch Capabilities Index	—	166.09

Math Not comparable

Devstral Small 2505: —, GPT-6.1 Sol: 93.7 (#1)

Math benchmarks
Benchmark	Devstral Small 2505	GPT-6.1 Sol
FrontierMath (Tiers 1-3)	—	93.7%
FrontierMath Tier 4	—	100%
OTIS Mock AIME 2024-2025	—	100%
ProofBench	—	99%
LMArena Math	—	1464

Knowledge Not comparable

Devstral Small 2505: —, GPT-6.1 Sol: 71.8 (#4)

Knowledge benchmarks
Benchmark	Devstral Small 2505	GPT-6.1 Sol
GPQA Diamond	—	95.4%
SimpleQA Verified	—	73.9%
LMArena Expert	—	1502

Multimodal Not comparable

Devstral Small 2505: —, GPT-6.1 Sol: 52.7 (#5)

Multimodal benchmarks
Benchmark	Devstral Small 2505	GPT-6.1 Sol
LMArena Vision	—	1288
Furniture Assembly	—	80%

Multilingual Not comparable

Devstral Small 2505: —, GPT-6.1 Sol: 54.3 (#46)

Multilingual benchmarks
Benchmark	Devstral Small 2505	GPT-6.1 Sol
LMArena Non-English	—	1438
LMArena Chinese	—	1477
LMArena Russian	—	1455

Instruction Following Not comparable

Devstral Small 2505: —, GPT-6.1 Sol: 77.0 (#29)

Instruction Following benchmarks
Benchmark	Devstral Small 2505	GPT-6.1 Sol
LMArena Instruction Following	—	1468

Long Context Not comparable

Devstral Small 2505: —, GPT-6.1 Sol: 44.9 (#54)

Long Context benchmarks
Benchmark	Devstral Small 2505	GPT-6.1 Sol
LMArena Longer Query	—	1465

Writing & Preference Not comparable

Devstral Small 2505: —, GPT-6.1 Sol: 63.6 (#63)

Writing & Preference benchmarks
Benchmark	Devstral Small 2505	GPT-6.1 Sol
LMArena Text	—	1447
LMArena Creative Writing	—	1432
LMArena Multi-Turn	—	1449

Frequently asked questions

Is Devstral Small 2505 better than GPT-6.1 Sol?

Which is cheaper, Devstral Small 2505 or GPT-6.1 Sol?

Devstral Small 2505 is cheaper. It lists at $0.10 per million input tokens and $0.30 per million output tokens; GPT-6.1 Sol lists at $2 and $10.

Is Devstral Small 2505 or GPT-6.1 Sol better for coding?

GPT-6.1 Sol scores higher on coding benchmarks: 63.2 versus 38.9 in the Noometry coding category.

Which has the bigger context window?

GPT-6.1 Sol does, with 1.05M tokens against 128K.

How many benchmarks do Devstral Small 2505 and GPT-6.1 Sol share?

2 benchmarks have published results for both models. Devstral Small 2505 has 4 scored results on Noometry and GPT-6.1 Sol has 34.