Noometry

Model comparison

Dolly 2.0-12b vs Llama-3.3-70B-Instruct

Llama-3.3-70B-Instruct is the stronger model overall, scoring 30.3 to 24.9 on the Noometry Index.

Last verified . 11 shared benchmarks.

Dolly 2.0-12b Databricks

24.9

Rank #339 Confirmed

Llama-3.3-70B-Instruct Meta

30.3

Rank #283 Confirmed

Summary

  • They share 11 benchmarks with published results for both. Dolly 2.0-12b scores higher in 2 categories and Llama-3.3-70B-Instruct in 4 categories; 5 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Llama-3.3-70B-Instruct leads 47.8 to 15.3.

Side by side

Dolly 2.0-12bLlama-3.3-70B-Instruct
ProviderDatabricksMeta
Noometry Index24.930.3
Released2023-04-112024-12-06
WeightsOpenOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.10
Output $ / M tokens—$0.32
Results tracked1743

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama-3.3-70B-Instruct leads

Dolly 2.0-12b: 23.0 (#327), Llama-3.3-70B-Instruct: 30.6 (#284)

BenchmarkDolly 2.0-12bLlama-3.3-70B-Instruct
LMArena Coding7761268
SciCode—26%
WeirdML—14.4%
BigCodeBench Instruct—46.9%
LiveBench Coding—36.6%
BigCodeBench Complete—57.5%

Agentic & Tool Use Not comparable

Dolly 2.0-12b: —, Llama-3.3-70B-Instruct: 25.6 (#100)

BenchmarkDolly 2.0-12bLlama-3.3-70B-Instruct
Berkeley Function Calling Leaderboard—31.9%
BALROG—23%

Reasoning Too close to call

Dolly 2.0-12b: 14.6 (#315), Llama-3.3-70B-Instruct: 13.7 (#325)

BenchmarkDolly 2.0-12bLlama-3.3-70B-Instruct
LMArena Hard Prompts8041257
Epoch Capabilities Index88.94127.32
SimpleBench—19.9%
CritPt—0%
LiveBench Reasoning—50.8%
DTBench—59.5%
LiveBench Data Analysis—49.5%
LMCA—17.5%
ForecastBench—58.6
HellaSwag70.8%—
LiveBench—50.2%
PIQA75.4%—
WinoGrande61.8%—

Math Dolly 2.0-12b leads

Dolly 2.0-12b: 25.0 (#250), Llama-3.3-70B-Instruct: 14.3 (#292)

BenchmarkDolly 2.0-12bLlama-3.3-70B-Instruct
LMArena Math8711268
OTIS Mock AIME 2024-2025—5.1%
LiveBench Math—42.2%
MATH Level 5—41.6%

Knowledge Not comparable

Dolly 2.0-12b: —, Llama-3.3-70B-Instruct: 30.5 (#220)

BenchmarkDolly 2.0-12bLlama-3.3-70B-Instruct
MMLU26.2%86.3%
GPQA Diamond—47.4%
Confabulations—22.8%
Vectara Hallucination Rate—4.1%
LMArena Expert—1225
ARC (AI2) Challenge39.6%—
BoolQ56.3%—
OpenBookQA39.2%—

Multilingual Llama-3.3-70B-Instruct leads

Dolly 2.0-12b: 17.4 (#291), Llama-3.3-70B-Instruct: 40.1 (#215)

BenchmarkDolly 2.0-12bLlama-3.3-70B-Instruct
LMArena Non-English8361236
LMArena Chinese8361217
LMArena French—1280
LMArena German—1251
LMArena Japanese—1149
LMArena Korean—1142
LMArena Russian—1252
LMArena Spanish—1269

Instruction Following Llama-3.3-70B-Instruct leads

Dolly 2.0-12b: 38.9 (#299), Llama-3.3-70B-Instruct: 71.3 (#152)

BenchmarkDolly 2.0-12bLlama-3.3-70B-Instruct
LMArena Instruction Following8141242
LiveBench Instruction Following—82.7%

Long Context Not comparable

Dolly 2.0-12b: —, Llama-3.3-70B-Instruct: 26.4 (#290)

BenchmarkDolly 2.0-12bLlama-3.3-70B-Instruct
Fiction.LiveBench—33.3%
LMArena Longer Query—1256

Writing & Preference Llama-3.3-70B-Instruct leads

Dolly 2.0-12b: 15.3 (#306), Llama-3.3-70B-Instruct: 47.8 (#203)

BenchmarkDolly 2.0-12bLlama-3.3-70B-Instruct
LMArena Text8511274
LMArena Creative Writing8641250
LMArena Multi-Turn7401279
LiveBench Language—39.2%

Frequently asked questions

Is Dolly 2.0-12b better than Llama-3.3-70B-Instruct?

Llama-3.3-70B-Instruct is the stronger model overall, scoring 30.3 to 24.9 on the Noometry Index.

Is Dolly 2.0-12b or Llama-3.3-70B-Instruct better for coding?

Llama-3.3-70B-Instruct scores higher on coding benchmarks: 30.6 versus 23.0 in the Noometry coding category.

How many benchmarks do Dolly 2.0-12b and Llama-3.3-70B-Instruct share?

11 benchmarks have published results for both models. Dolly 2.0-12b has 17 scored results on Noometry and Llama-3.3-70B-Instruct has 43.

Related comparisons

Go deeper