Model comparison

DeepSeek LLM 67B vs Mistral Nemo

Mistral Nemo is the stronger model overall, scoring 26.4 to 24.9 on the Noometry Index.

Last verified . 3 shared benchmarks.

DeepSeek LLM 67B DeepSeek

24.9

Rank #347 Confirmed

Mistral Nemo Mistral AI

26.4

Rank #337 Confirmed

Summary

  • They share 3 benchmarks with published results for both. DeepSeek LLM 67B scores higher in 1 category and Mistral Nemo in 3 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in math, where Mistral Nemo leads 25.5 to 8.7.
  • The biggest single-benchmark swing is GPQA Diamond: 24.6% for DeepSeek LLM 67B and 29.9% for Mistral Nemo.

Side by side

DeepSeek LLM 67B and Mistral Nemo specifications
DeepSeek LLM 67BMistral Nemo
ProviderDeepSeekMistral AI
Noometry Index24.926.4
Released2023-11-292024-07-01
WeightsOpenOpen
Context window—128K
Max output—128K
Input $ / M tokens—$0.15
Output $ / M tokens—$0.15
Results tracked1510

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

DeepSeek LLM 67B: 31.9 (#278), Mistral Nemo: —

Coding benchmarks
BenchmarkDeepSeek LLM 67BMistral Nemo
LMArena Coding1096—

Agentic & Tool Use Not comparable

DeepSeek LLM 67B: —, Mistral Nemo: 23.5 (#125)

Agentic & Tool Use benchmarks
BenchmarkDeepSeek LLM 67BMistral Nemo
Berkeley Function Calling Leaderboard—27.6%
BALROG—17.6%

Reasoning Mistral Nemo leads

DeepSeek LLM 67B: 16.5 (#304), Mistral Nemo: 20.7 (#232)

Reasoning benchmarks
BenchmarkDeepSeek LLM 67BMistral Nemo
Epoch Capabilities Index110.5118.68
Chess Puzzles0%—
LMArena Hard Prompts1070—
DTBench—48.6%
PIQA—83.5%

Math Mistral Nemo leads

DeepSeek LLM 67B: 8.7 (#324), Mistral Nemo: 25.5 (#268)

Math benchmarks
BenchmarkDeepSeek LLM 67BMistral Nemo
MATH Level 56.4%10.8%
OTIS Mock AIME 2024-20250.8%—
LMArena Math1108—
GSM8K—84.2%

Knowledge Mistral Nemo leads

DeepSeek LLM 67B: 7.0 (#313), Mistral Nemo: 12.3 (#298)

Knowledge benchmarks
BenchmarkDeepSeek LLM 67BMistral Nemo
GPQA Diamond24.6%29.9%
BoolQ—82.5%

Multilingual Not comparable

DeepSeek LLM 67B: 29.4 (#267), Mistral Nemo: —

Multilingual benchmarks
BenchmarkDeepSeek LLM 67BMistral Nemo
LMArena Non-English1073—
LMArena Chinese1132—

Instruction Following Not comparable

DeepSeek LLM 67B: 55.4 (#277), Mistral Nemo: —

Instruction Following benchmarks
BenchmarkDeepSeek LLM 67BMistral Nemo
LMArena Instruction Following1079—

Long Context Not comparable

DeepSeek LLM 67B: 33.1 (#265), Mistral Nemo: —

Long Context benchmarks
BenchmarkDeepSeek LLM 67BMistral Nemo
LMArena Longer Query1092—

Writing & Preference DeepSeek LLM 67B leads

DeepSeek LLM 67B: 31.6 (#282), Mistral Nemo: 28.5 (#296)

Writing & Preference benchmarks
BenchmarkDeepSeek LLM 67BMistral Nemo
LMArena Text1105—
LMArena Creative Writing1067—
EQ-Bench Creative Writing—881
LMArena Multi-Turn1082—

Frequently asked questions

Is DeepSeek LLM 67B better than Mistral Nemo?

Mistral Nemo is the stronger model overall, scoring 26.4 to 24.9 on the Noometry Index.

How many benchmarks do DeepSeek LLM 67B and Mistral Nemo share?

3 benchmarks have published results for both models. DeepSeek LLM 67B has 15 scored results on Noometry and Mistral Nemo has 10.

Related comparisons

Go deeper