Model comparison
DeepSeek Coder 6.7B vs Olmo 7b Instruct
Olmo 7b Instruct has enough public results to be ranked (#295); DeepSeek Coder 6.7B does not yet, so treat this comparison as directional.
Last verified . 0 shared benchmarks.
DeepSeek
Allen Institute for AI (Ai2)Summary
- The widest gap is in coding, where DeepSeek Coder 6.7B leads 35.8 to 29.6.
Side by side
| DeepSeek Coder 6.7B | Olmo 7b Instruct | |
|---|---|---|
| Provider | DeepSeek | Allen Institute for AI (Ai2) |
| Noometry Index | 37.6 | 30.3 |
| Released | 2023-11-02 | — |
| Weights | Open | Open |
| Context window | — | — |
| Max output | — | — |
| Input $ / M tokens | — | — |
| Output $ / M tokens | — | — |
| Results tracked | 9 | 10 |
Sponsored placements are available on pages like this one. Advertise on Noometry
Category by category
Coding DeepSeek Coder 6.7B leads
DeepSeek Coder 6.7B: 35.8 (#219), Olmo 7b Instruct: 29.6 (#303)
| Benchmark | DeepSeek Coder 6.7B | Olmo 7b Instruct |
|---|---|---|
| BigCodeBench Instruct | 35.5% | — |
| LMArena Coding | — | 1016 |
| BigCodeBench Complete | 43.8% | — |
| HumanEval+ | 71.3% | — |
| MBPP+ | 65.6% | — |
Reasoning Not comparable
DeepSeek Coder 6.7B: —, Olmo 7b Instruct: 18.8 (#274)
| Benchmark | DeepSeek Coder 6.7B | Olmo 7b Instruct |
|---|---|---|
| LMArena Hard Prompts | — | 993 |
| Epoch Capabilities Index | 89.62 | — |
| WinoGrande | 57.6% | — |
Math Not comparable
DeepSeek Coder 6.7B: —, Olmo 7b Instruct: 30.2 (#237)
| Benchmark | DeepSeek Coder 6.7B | Olmo 7b Instruct |
|---|---|---|
| LMArena Math | — | 1018 |
| GSM8K | 21.3% | — |
Knowledge Not comparable
DeepSeek Coder 6.7B: —, Olmo 7b Instruct: —
| Benchmark | DeepSeek Coder 6.7B | Olmo 7b Instruct |
|---|---|---|
| ARC (AI2) Challenge | 36.4% | — |
| MMLU | 36.4% | — |
Multilingual Not comparable
DeepSeek Coder 6.7B: —, Olmo 7b Instruct: 24.0 (#291)
| Benchmark | DeepSeek Coder 6.7B | Olmo 7b Instruct |
|---|---|---|
| LMArena Non-English | — | 977 |
| LMArena Chinese | — | 1014 |
| LMArena Russian | — | 947 |
Instruction Following Not comparable
DeepSeek Coder 6.7B: —, Olmo 7b Instruct: 49.0 (#301)
| Benchmark | DeepSeek Coder 6.7B | Olmo 7b Instruct |
|---|---|---|
| LMArena Instruction Following | — | 978 |
Writing & Preference Not comparable
DeepSeek Coder 6.7B: —, Olmo 7b Instruct: 25.8 (#303)
| Benchmark | DeepSeek Coder 6.7B | Olmo 7b Instruct |
|---|---|---|
| LMArena Text | — | 1032 |
| LMArena Creative Writing | — | 990 |
| LMArena Multi-Turn | — | 1007 |
Frequently asked questions
Is DeepSeek Coder 6.7B better than Olmo 7b Instruct?
Olmo 7b Instruct has enough public results to be ranked (#295); DeepSeek Coder 6.7B does not yet, so treat this comparison as directional.
Is DeepSeek Coder 6.7B or Olmo 7b Instruct better for coding?
DeepSeek Coder 6.7B scores higher on coding benchmarks: 35.8 versus 29.6 in the Noometry coding category.
How many benchmarks do DeepSeek Coder 6.7B and Olmo 7b Instruct share?
0 benchmarks have published results for both models. DeepSeek Coder 6.7B has 9 scored results on Noometry and Olmo 7b Instruct has 10.