Noometry

Model comparison

Claude 2 vs DeepSeek Coder 6.7B

Claude 2 has enough public results to be ranked (#346); DeepSeek Coder 6.7B does not yet, so treat this comparison as directional.

Last verified . 3 shared benchmarks.

Claude 2 Anthropic

25.0

Rank #346 Reported

DeepSeek Coder 6.7B DeepSeek

37.6

Unranked Sparse

Summary

  • They share 3 benchmarks with published results for both.
  • DeepSeek Coder 6.7B has downloadable open weights; the other is API-only.

Side by side

Claude 2DeepSeek Coder 6.7B
ProviderAnthropicDeepSeek
Noometry Index25.037.6
Released2023-07-112023-11-02
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked89

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Claude 2: —, DeepSeek Coder 6.7B: 35.8 (#219)

BenchmarkClaude 2DeepSeek Coder 6.7B
HumanEval+61.6%71.3%
BigCodeBench Instruct—35.5%
BigCodeBench Complete—43.8%
MBPP+—65.6%

Reasoning Not comparable

Claude 2: 21.7 (#216), DeepSeek Coder 6.7B: —

BenchmarkClaude 2DeepSeek Coder 6.7B
Epoch Capabilities Index120.1389.62
DTBench51.9%—
WinoGrande—57.6%

Math Not comparable

Claude 2: 9.3 (#320), DeepSeek Coder 6.7B: —

BenchmarkClaude 2DeepSeek Coder 6.7B
OTIS Mock AIME 2024-20252.5%—
MATH Level 511.7%—
GSM8K—21.3%

Knowledge Not comparable

Claude 2: 16.9 (#287), DeepSeek Coder 6.7B: —

BenchmarkClaude 2DeepSeek Coder 6.7B
MMLU78.5%36.4%
GPQA Diamond34.7%—
ARC (AI2) Challenge—36.4%
TriviaQA87.5%—

Frequently asked questions

Is Claude 2 better than DeepSeek Coder 6.7B?

Claude 2 has enough public results to be ranked (#346); DeepSeek Coder 6.7B does not yet, so treat this comparison as directional.

How many benchmarks do Claude 2 and DeepSeek Coder 6.7B share?

3 benchmarks have published results for both models. Claude 2 has 8 scored results on Noometry and DeepSeek Coder 6.7B has 9.

Related comparisons

Go deeper