Noometry

Model comparison

Devstral Small 2505 vs Llama 2-34B

Devstral Small 2505 has enough public results to be ranked (#233); Llama 2-34B does not yet, so treat this comparison as directional.

Last verified . 0 shared benchmarks.

Devstral Small 2505 Mistral AI

34.3

Rank #233 Reported

Llama 2-34B Meta

—

Unranked

Summary

  • Devstral Small 2505 has downloadable open weights; the other is API-only.

Side by side

Devstral Small 2505Llama 2-34B
ProviderMistral AIMeta
Noometry Index34.3—
Released2025-05-072023-07-18
WeightsOpenProprietary
Context window128K—
Max output128K—
Input $ / M tokens$0.10—
Output $ / M tokens$0.30—
Results tracked410

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Devstral Small 2505: 38.9 (#166), Llama 2-34B: —

BenchmarkDevstral Small 2505Llama 2-34B
SWE-bench Verified (bash only)56.4%—
SciCode28.8%—

Reasoning Not comparable

Devstral Small 2505: 19.7 (#252), Llama 2-34B: —

BenchmarkDevstral Small 2505Llama 2-34B
Kagi LLM Benchmark37.7%—
CritPt0%—
BIG-Bench Hard—44.1%
Epoch Capabilities Index—105.2
PIQA—81.9%
WinoGrande—76.7%

Math Not comparable

Devstral Small 2505: —, Llama 2-34B: —

BenchmarkDevstral Small 2505Llama 2-34B
GSM8K—42.2%

Knowledge Not comparable

Devstral Small 2505: —, Llama 2-34B: —

BenchmarkDevstral Small 2505Llama 2-34B
ARC (AI2) Challenge—54.5%
BoolQ—83.7%
MMLU—62.6%
OpenBookQA—58.2%
TriviaQA—84.6%

Frequently asked questions

Is Devstral Small 2505 better than Llama 2-34B?

Devstral Small 2505 has enough public results to be ranked (#233); Llama 2-34B does not yet, so treat this comparison as directional.

How many benchmarks do Devstral Small 2505 and Llama 2-34B share?

0 benchmarks have published results for both models. Devstral Small 2505 has 4 scored results on Noometry and Llama 2-34B has 10.

Related comparisons

Go deeper