Noometry

Model comparison

Claude 3 Sonnet vs Llama-3.3-70B-Instruct

Llama-3.3-70B-Instruct is the stronger model overall, scoring 30.3 to 28.8 on the Noometry Index.

Last verified . 26 shared benchmarks.

Claude 3 Sonnet Anthropic

28.8

Rank #311 Confirmed

Llama-3.3-70B-Instruct Meta

30.3

Rank #283 Confirmed

Summary

  • They share 26 benchmarks with published results for both. Claude 3 Sonnet scores higher in 2 categories and Llama-3.3-70B-Instruct in 6 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in long context, where Claude 3 Sonnet leads 36.7 to 26.4.
  • The biggest single-benchmark swing is MATH Level 5: 18.2% for Claude 3 Sonnet and 41.6% for Llama-3.3-70B-Instruct.
  • Llama-3.3-70B-Instruct has downloadable open weights; the other is API-only.

Side by side

Claude 3 SonnetLlama-3.3-70B-Instruct
ProviderAnthropicMeta
Noometry Index28.830.3
Released2024-02-292024-12-06
WeightsProprietaryOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.10
Output $ / M tokens—$0.32
Results tracked3043

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama-3.3-70B-Instruct leads

Claude 3 Sonnet: 29.2 (#296), Llama-3.3-70B-Instruct: 30.6 (#284)

BenchmarkClaude 3 SonnetLlama-3.3-70B-Instruct
WeirdML10.2%14.4%
BigCodeBench Instruct42.7%46.9%
LMArena Coding12231268
BigCodeBench Complete53.8%57.5%
SciCode—26%
LiveBench Coding—36.6%
HumanEval+64%—
MBPP+69.3%—

Agentic & Tool Use Not comparable

Claude 3 Sonnet: —, Llama-3.3-70B-Instruct: 25.6 (#100)

BenchmarkClaude 3 SonnetLlama-3.3-70B-Instruct
Berkeley Function Calling Leaderboard—31.9%
BALROG—23%

Reasoning Claude 3 Sonnet leads

Claude 3 Sonnet: 19.7 (#232), Llama-3.3-70B-Instruct: 13.7 (#325)

BenchmarkClaude 3 SonnetLlama-3.3-70B-Instruct
LMArena Hard Prompts11971257
DTBench53.6%59.5%
Epoch Capabilities Index120.69127.32
SimpleBench—19.9%
CritPt—0%
LiveBench Reasoning—50.8%
LiveBench Data Analysis—49.5%
LMCA—17.5%
ForecastBench—58.6
LiveBench—50.2%
WinoGrande75.1%—

Math Llama-3.3-70B-Instruct leads

Claude 3 Sonnet: 10.0 (#305), Llama-3.3-70B-Instruct: 14.3 (#292)

BenchmarkClaude 3 SonnetLlama-3.3-70B-Instruct
OTIS Mock AIME 2024-20252.5%5.1%
LMArena Math12131268
MATH Level 518.2%41.6%
LiveBench Math—42.2%

Knowledge Llama-3.3-70B-Instruct leads

Claude 3 Sonnet: 21.1 (#271), Llama-3.3-70B-Instruct: 30.5 (#220)

BenchmarkClaude 3 SonnetLlama-3.3-70B-Instruct
GPQA Diamond40.6%47.4%
LMArena Expert11731225
MMLU75.9%86.3%
Confabulations—22.8%
Vectara Hallucination Rate—4.1%

Multimodal Not comparable

Claude 3 Sonnet: 24.7 (#120), Llama-3.3-70B-Instruct: —

BenchmarkClaude 3 SonnetLlama-3.3-70B-Instruct
LMArena Vision984—

Multilingual Llama-3.3-70B-Instruct leads

Claude 3 Sonnet: 38.0 (#229), Llama-3.3-70B-Instruct: 40.1 (#215)

BenchmarkClaude 3 SonnetLlama-3.3-70B-Instruct
LMArena Non-English12051236
LMArena Chinese11881217
LMArena French12281280
LMArena German12041251
LMArena Japanese11311149
LMArena Korean11281142
LMArena Russian12271252
LMArena Spanish12041269

Instruction Following Llama-3.3-70B-Instruct leads

Claude 3 Sonnet: 63.1 (#230), Llama-3.3-70B-Instruct: 71.3 (#152)

BenchmarkClaude 3 SonnetLlama-3.3-70B-Instruct
LMArena Instruction Following12001242
LiveBench Instruction Following—82.7%

Long Context Claude 3 Sonnet leads

Claude 3 Sonnet: 36.7 (#223), Llama-3.3-70B-Instruct: 26.4 (#290)

BenchmarkClaude 3 SonnetLlama-3.3-70B-Instruct
LMArena Longer Query12111256
Fiction.LiveBench—33.3%

Writing & Preference Llama-3.3-70B-Instruct leads

Claude 3 Sonnet: 42.3 (#233), Llama-3.3-70B-Instruct: 47.8 (#203)

BenchmarkClaude 3 SonnetLlama-3.3-70B-Instruct
LMArena Text12181274
LMArena Creative Writing11861250
LMArena Multi-Turn12271279
LiveBench Language—39.2%

Frequently asked questions

Is Claude 3 Sonnet better than Llama-3.3-70B-Instruct?

Llama-3.3-70B-Instruct is the stronger model overall, scoring 30.3 to 28.8 on the Noometry Index.

Is Claude 3 Sonnet or Llama-3.3-70B-Instruct better for coding?

Llama-3.3-70B-Instruct scores higher on coding benchmarks: 30.6 versus 29.2 in the Noometry coding category.

How many benchmarks do Claude 3 Sonnet and Llama-3.3-70B-Instruct share?

26 benchmarks have published results for both models. Claude 3 Sonnet has 30 scored results on Noometry and Llama-3.3-70B-Instruct has 43.

Related comparisons

Go deeper