Noometry

Model comparison

DeepSeek-V2.5 (Sep 2024) vs Pixtral Large

DeepSeek-V2.5 (Sep 2024) is the stronger model overall, scoring 37.6 to 32.2 on the Noometry Index.

Last verified . 0 shared benchmarks.

DeepSeek-V2.5 (Sep 2024) DeepSeek

37.6

Rank #200 Confirmed

Pixtral Large Mistral AI

32.2

Rank #259 Reported

Summary

  • The widest gap is in writing & preference, where DeepSeek-V2.5 (Sep 2024) leads 49.8 to 32.9.

Side by side

DeepSeek-V2.5 (Sep 2024)Pixtral Large
ProviderDeepSeekMistral AI
Noometry Index37.632.2
Released2024-09-062024-11-01
WeightsOpenOpen
Context window—128K
Max output—128K
Input $ / M tokens—$2
Output $ / M tokens—$6
Results tracked223

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

DeepSeek-V2.5 (Sep 2024): 31.7 (#281), Pixtral Large: —

BenchmarkDeepSeek-V2.5 (Sep 2024)Pixtral Large
Aider Polyglot17.8%—
BigCodeBench Instruct48.6%—
LMArena Coding1309—
BigCodeBench Complete53.2%—
HumanEval+83.5%—
MBPP+74.1%—

Reasoning DeepSeek-V2.5 (Sep 2024) leads

DeepSeek-V2.5 (Sep 2024): 25.6 (#145), Pixtral Large: 21.7 (#218)

BenchmarkDeepSeek-V2.5 (Sep 2024)Pixtral Large
EnigmaEval—0.8%
LMArena Hard Prompts1289—

Math Not comparable

DeepSeek-V2.5 (Sep 2024): 35.9 (#177), Pixtral Large: —

BenchmarkDeepSeek-V2.5 (Sep 2024)Pixtral Large
LMArena Math1288—

Knowledge Not comparable

DeepSeek-V2.5 (Sep 2024): 34.8 (#193), Pixtral Large: —

BenchmarkDeepSeek-V2.5 (Sep 2024)Pixtral Large
LMArena Expert1266—

Multimodal Not comparable

DeepSeek-V2.5 (Sep 2024): —, Pixtral Large: 30.6 (#111)

BenchmarkDeepSeek-V2.5 (Sep 2024)Pixtral Large
LMArena Vision—1089

Multilingual Not comparable

DeepSeek-V2.5 (Sep 2024): 42.5 (#193), Pixtral Large: —

BenchmarkDeepSeek-V2.5 (Sep 2024)Pixtral Large
LMArena Non-English1273—
LMArena Chinese1318—
LMArena French1289—
LMArena German1258—
LMArena Japanese1228—
LMArena Korean1209—
LMArena Russian1289—
LMArena Spanish1248—

Instruction Following Not comparable

DeepSeek-V2.5 (Sep 2024): 67.5 (#194), Pixtral Large: —

BenchmarkDeepSeek-V2.5 (Sep 2024)Pixtral Large
LMArena Instruction Following1280—

Long Context Not comparable

DeepSeek-V2.5 (Sep 2024): 39.5 (#174), Pixtral Large: —

BenchmarkDeepSeek-V2.5 (Sep 2024)Pixtral Large
LMArena Longer Query1301—

Writing & Preference DeepSeek-V2.5 (Sep 2024) leads

DeepSeek-V2.5 (Sep 2024): 49.8 (#187), Pixtral Large: 32.9 (#278)

BenchmarkDeepSeek-V2.5 (Sep 2024)Pixtral Large
LMArena Text1294—
LMArena Creative Writing1285—
EQ-Bench Creative Writing—988
LMArena Multi-Turn1297—

Frequently asked questions

Is DeepSeek-V2.5 (Sep 2024) better than Pixtral Large?

DeepSeek-V2.5 (Sep 2024) is the stronger model overall, scoring 37.6 to 32.2 on the Noometry Index.

How many benchmarks do DeepSeek-V2.5 (Sep 2024) and Pixtral Large share?

0 benchmarks have published results for both models. DeepSeek-V2.5 (Sep 2024) has 22 scored results on Noometry and Pixtral Large has 3.

Related comparisons

Go deeper