Noometry

Model comparison

Devstral Small 2505 vs GPT-4 Turbo

Devstral Small 2505 is the stronger model overall, scoring 34.3 to 30.5 on the Noometry Index.

Last verified . 0 shared benchmarks.

Devstral Small 2505 Mistral AI

34.3

Rank #233 Reported

GPT-4 Turbo OpenAI

30.5

Rank #292 Confirmed

Summary

  • The widest gap is in coding, where Devstral Small 2505 leads 38.9 to 33.8.
  • Devstral Small 2505 is cheaper at $0.10 / $0.30 per million input/output tokens, against $10 / $30 for GPT-4 Turbo.
  • Devstral Small 2505 has downloadable open weights; the other is API-only.

Side by side

Devstral Small 2505GPT-4 Turbo
ProviderMistral AIOpenAI
Noometry Index34.330.5
Released2025-05-072023-11-06
WeightsOpenProprietary
Context window128K128K
Max output128K4K
Input $ / M tokens$0.10$10
Output $ / M tokens$0.30$30
Results tracked436

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Devstral Small 2505 leads

Devstral Small 2505: 38.9 (#166), GPT-4 Turbo: 33.8 (#249)

BenchmarkDevstral Small 2505GPT-4 Turbo
SWE-bench Verified (bash only)56.4%—
SciCode28.8%—
WeirdML—18%
BigCodeBench Instruct—48.2%
LMArena Coding—1268
BigCodeBench Complete—58.2%
HumanEval+—86.6%
MBPP+—73.3%

Agentic & Tool Use Not comparable

Devstral Small 2505: —, GPT-4 Turbo: —

BenchmarkDevstral Small 2505GPT-4 Turbo
METR Time Horizons—36.7%

Reasoning Devstral Small 2505 leads

Devstral Small 2505: 19.7 (#252), GPT-4 Turbo: 15.3 (#317)

BenchmarkDevstral Small 2505GPT-4 Turbo
SimpleBench—25.1%
Kagi LLM Benchmark37.7%—
CritPt0%—
Chess Puzzles—6%
LMArena Hard Prompts—1251
DTBench—61.6%
LMCA—9.8%
Epoch Capabilities Index—127.25
ForecastBench—59.4

Math Not comparable

Devstral Small 2505: —, GPT-4 Turbo: 9.0 (#322)

BenchmarkDevstral Small 2505GPT-4 Turbo
FrontierMath (Tiers 1-3)—0.7%
OTIS Mock AIME 2024-2025—6.7%
LMArena Math—1272
MATH Level 5—46.7%

Knowledge Not comparable

Devstral Small 2505: —, GPT-4 Turbo: 24.3 (#268)

BenchmarkDevstral Small 2505GPT-4 Turbo
GPQA Diamond—46.6%
Confabulations—28.4%
LMArena Expert—1223
MMLU—81.3%

Multimodal Not comparable

Devstral Small 2505: —, GPT-4 Turbo: 30.6 (#110)

BenchmarkDevstral Small 2505GPT-4 Turbo
LMArena Vision—1090

Multilingual Not comparable

Devstral Small 2505: —, GPT-4 Turbo: 40.5 (#216)

BenchmarkDevstral Small 2505GPT-4 Turbo
LMArena Non-English—1245
LMArena Chinese—1242
LMArena French—1276
LMArena German—1259
LMArena Japanese—1194
LMArena Korean—1187
LMArena Russian—1259
LMArena Spanish—1260

Instruction Following Not comparable

Devstral Small 2505: —, GPT-4 Turbo: 65.8 (#216)

BenchmarkDevstral Small 2505GPT-4 Turbo
LMArena Instruction Following—1249

Long Context Not comparable

Devstral Small 2505: —, GPT-4 Turbo: 38.0 (#206)

BenchmarkDevstral Small 2505GPT-4 Turbo
LMArena Longer Query—1254

Writing & Preference Not comparable

Devstral Small 2505: —, GPT-4 Turbo: 47.7 (#206)

BenchmarkDevstral Small 2505GPT-4 Turbo
LMArena Text—1272
LMArena Creative Writing—1269
LMArena Multi-Turn—1267

Frequently asked questions

Is Devstral Small 2505 better than GPT-4 Turbo?

Devstral Small 2505 is the stronger model overall, scoring 34.3 to 30.5 on the Noometry Index.

Which is cheaper, Devstral Small 2505 or GPT-4 Turbo?

Devstral Small 2505 is cheaper. It lists at $0.10 per million input tokens and $0.30 per million output tokens; GPT-4 Turbo lists at $10 and $30.

Is Devstral Small 2505 or GPT-4 Turbo better for coding?

Devstral Small 2505 scores higher on coding benchmarks: 38.9 versus 33.8 in the Noometry coding category.

Which has the bigger context window?

Both accept 128K tokens.

How many benchmarks do Devstral Small 2505 and GPT-4 Turbo share?

0 benchmarks have published results for both models. Devstral Small 2505 has 4 scored results on Noometry and GPT-4 Turbo has 36.

Related comparisons

Go deeper