DeepSeek, open weights

DeepSeek Coder 33B

DeepSeek Coder 33B by DeepSeek has 9 published benchmark results on Noometry, not yet enough to be ranked.

Last verified

Specifications

Noometry rank
Unranked
Index score
Not ranked
Evidence
Sparse 9 results
Provider
DeepSeek
Released
November 2, 2023
Weights
Open weights
Reasoning
Unknown
Context window
—
Max output
—
Input price
Not listed
Output price
Not listed
Blended price
Not listed
Output speed
Not measured
Value
Not ranked
Knowledge cutoff
Unknown

Category scores

Each category score combines every public result we have in that category.

DeepSeek Coder 33B category scores
  1. Coding 38.0
DeepSeek Coder 33B category ranks
CategoryScoreRankResults
Coding38.0#1842

Sponsored placements are available on pages like this one. Advertise on Noometry

Benchmark results

Every published result we track, with its source. Bold rows are the ones used for ranking; where several exist we prefer independent runs over self-reported numbers.

Coding

DeepSeek Coder 33B Coding benchmark results
BenchmarkScorePositionSettingSourceDate
BigCodeBench Instruct42%#29 of 64, top 46%BigCodeBench2023-10-28
BigCodeBench Complete46.6%BigCodeBench2023-10-28
BigCodeBench Complete51.1%#30 of 66, top 46%BigCodeBench2023-10-28
HumanEval+44.5%EvalPlus
HumanEval+75%#15 of 45, top 34%EvalPlus
MBPP+70.1%#13 of 38, top 35%EvalPlus

Reasoning

DeepSeek Coder 33B Reasoning benchmark results
BenchmarkScorePositionSettingSourceDate
Epoch Capabilities Index96.32#205 of 213, top 97%Epoch AI2023-11-02
WinoGrande62%#37 of 43, top 87%Epoch AI

Math

DeepSeek Coder 33B Math benchmark results
BenchmarkScorePositionSettingSourceDate
GSM8K35.4%#29 of 38, top 77%Epoch AI

Knowledge

DeepSeek Coder 33B Knowledge benchmark results
BenchmarkScorePositionSettingSourceDate
ARC (AI2) Challenge42.2%#33 of 39, top 85%Epoch AI
MMLU39.4%#74 of 81, top 92%Epoch AI

Compare DeepSeek Coder 33B

Other DeepSeek models

Frequently asked questions

How good is DeepSeek Coder 33B?

DeepSeek Coder 33B by DeepSeek has 9 published benchmark results on Noometry, not yet enough to be ranked.

Is DeepSeek Coder 33B open source?

Yes. DeepSeek Coder 33B's weights are downloadable; check the license for commercial terms.