Noometry

Model comparison

DeepSeek-V2.5 (Sep 2024) vs Gemini 1.5 Flash 8B

DeepSeek-V2.5 (Sep 2024) is the stronger model overall, scoring 37.6 to 29.9 on the Noometry Index.

Last verified . 17 shared benchmarks.

DeepSeek-V2.5 (Sep 2024) DeepSeek

37.6

Rank #200 Confirmed

Gemini 1.5 Flash 8B Google

29.9

Rank #301 Confirmed

Summary

  • They share 17 benchmarks with published results for both. DeepSeek-V2.5 (Sep 2024) scores higher in 7 categories and Gemini 1.5 Flash 8B in 1 category; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where DeepSeek-V2.5 (Sep 2024) leads 35.9 to 14.2.
  • DeepSeek-V2.5 (Sep 2024) has downloadable open weights; the other is API-only.

Side by side

DeepSeek-V2.5 (Sep 2024)Gemini 1.5 Flash 8B
ProviderDeepSeekGoogle
Noometry Index37.629.9
Released2024-09-062024-10-03
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked2221

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 1.5 Flash 8B leads

DeepSeek-V2.5 (Sep 2024): 31.7 (#281), Gemini 1.5 Flash 8B: 35.5 (#225)

BenchmarkDeepSeek-V2.5 (Sep 2024)Gemini 1.5 Flash 8B
LMArena Coding13091218
Aider Polyglot17.8%—
BigCodeBench Instruct48.6%—
BigCodeBench Complete53.2%—
HumanEval+83.5%—
MBPP+74.1%—

Reasoning DeepSeek-V2.5 (Sep 2024) leads

DeepSeek-V2.5 (Sep 2024): 25.6 (#145), Gemini 1.5 Flash 8B: 20.0 (#244)

BenchmarkDeepSeek-V2.5 (Sep 2024)Gemini 1.5 Flash 8B
LMArena Hard Prompts12891209
DTBench—50%

Math DeepSeek-V2.5 (Sep 2024) leads

DeepSeek-V2.5 (Sep 2024): 35.9 (#177), Gemini 1.5 Flash 8B: 14.2 (#302)

BenchmarkDeepSeek-V2.5 (Sep 2024)Gemini 1.5 Flash 8B
LMArena Math12881207
OTIS Mock AIME 2024-2025—4.6%

Knowledge DeepSeek-V2.5 (Sep 2024) leads

DeepSeek-V2.5 (Sep 2024): 34.8 (#193), Gemini 1.5 Flash 8B: 16.0 (#289)

BenchmarkDeepSeek-V2.5 (Sep 2024)Gemini 1.5 Flash 8B
LMArena Expert12661185
GPQA Diamond—33%

Multimodal Not comparable

DeepSeek-V2.5 (Sep 2024): —, Gemini 1.5 Flash 8B: 28.2 (#115)

BenchmarkDeepSeek-V2.5 (Sep 2024)Gemini 1.5 Flash 8B
LMArena Vision—1044

Multilingual DeepSeek-V2.5 (Sep 2024) leads

DeepSeek-V2.5 (Sep 2024): 42.5 (#193), Gemini 1.5 Flash 8B: 38.5 (#229)

BenchmarkDeepSeek-V2.5 (Sep 2024)Gemini 1.5 Flash 8B
LMArena Non-English12731215
LMArena Chinese13181231
LMArena French12891234
LMArena German12581206
LMArena Japanese12281150
LMArena Korean12091140
LMArena Russian12891236
LMArena Spanish12481212

Instruction Following DeepSeek-V2.5 (Sep 2024) leads

DeepSeek-V2.5 (Sep 2024): 67.5 (#194), Gemini 1.5 Flash 8B: 62.8 (#236)

BenchmarkDeepSeek-V2.5 (Sep 2024)Gemini 1.5 Flash 8B
LMArena Instruction Following12801199

Long Context DeepSeek-V2.5 (Sep 2024) leads

DeepSeek-V2.5 (Sep 2024): 39.5 (#174), Gemini 1.5 Flash 8B: 37.0 (#225)

BenchmarkDeepSeek-V2.5 (Sep 2024)Gemini 1.5 Flash 8B
LMArena Longer Query13011219

Writing & Preference DeepSeek-V2.5 (Sep 2024) leads

DeepSeek-V2.5 (Sep 2024): 49.8 (#187), Gemini 1.5 Flash 8B: 42.8 (#232)

BenchmarkDeepSeek-V2.5 (Sep 2024)Gemini 1.5 Flash 8B
LMArena Text12941226
LMArena Creative Writing12851218
LMArena Multi-Turn12971185

Frequently asked questions

Is DeepSeek-V2.5 (Sep 2024) better than Gemini 1.5 Flash 8B?

DeepSeek-V2.5 (Sep 2024) is the stronger model overall, scoring 37.6 to 29.9 on the Noometry Index.

Is DeepSeek-V2.5 (Sep 2024) or Gemini 1.5 Flash 8B better for coding?

Gemini 1.5 Flash 8B scores higher on coding benchmarks: 35.5 versus 31.7 in the Noometry coding category.

How many benchmarks do DeepSeek-V2.5 (Sep 2024) and Gemini 1.5 Flash 8B share?

17 benchmarks have published results for both models. DeepSeek-V2.5 (Sep 2024) has 22 scored results on Noometry and Gemini 1.5 Flash 8B has 21.

Related comparisons

Go deeper