Noometry

Model comparison

Claude 3.5 Sonnet vs Llama 3.2 90B

Claude 3.5 Sonnet is the stronger model overall, scoring 34.6 to 27.5 on the Noometry Index.

Last verified . 9 shared benchmarks.

Claude 3.5 Sonnet Anthropic

34.6

Rank #231 Confirmed

Llama 3.2 90B Meta

27.5

Rank #331 Confirmed

Summary

  • They share 9 benchmarks with published results for both. Claude 3.5 Sonnet scores higher in 5 categories and Llama 3.2 90B in 0 categories; 5 gaps are clear of the uncertainty.
  • The widest gap is in math, where Claude 3.5 Sonnet leads 19.2 to 11.1.
  • The biggest single-benchmark swing is MATH Level 5: 56.9% for Claude 3.5 Sonnet and 39.4% for Llama 3.2 90B.
  • Llama 3.2 90B has downloadable open weights; the other is API-only.

Side by side

Claude 3.5 SonnetLlama 3.2 90B
ProviderAnthropicMeta
Noometry Index34.627.5
Released2024-06-202024-09-24
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked609

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Claude 3.5 Sonnet: 39.0 (#165), Llama 3.2 90B: —

BenchmarkClaude 3.5 SonnetLlama 3.2 90B
Aider Polyglot51.6%—
GSO4.6%—
WeirdML40%—
BigCodeBench Instruct46.8%—
LiveBench Coding67.1%—
LMArena Coding1342—
BigCodeBench Complete58.6%—
CadEval48%—
HumanEval+81.7%—
MBPP+74.3%—

Agentic & Tool Use Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 32.3 (#67), Llama 3.2 90B: 30.0 (#80)

BenchmarkClaude 3.5 SonnetLlama 3.2 90B
BALROG32.6%27.3%
TheAgentCompany24%—
Cybench17.5%—
METR Time Horizons45.2%—

Reasoning Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 23.1 (#183), Llama 3.2 90B: 21.7 (#217)

BenchmarkClaude 3.5 SonnetLlama 3.2 90B
EnigmaEval0.9%0.4%
Epoch Capabilities Index133.55125.5
SimpleBench41.4%—
LiveBench Reasoning56.7%—
LMArena Hard Prompts1305—
DTBench67.8%—
LiveBench Data Analysis55%—
ForecastBench60.7—
LiveBench59%—

Math Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 19.2 (#288), Llama 3.2 90B: 11.1 (#308)

BenchmarkClaude 3.5 SonnetLlama 3.2 90B
OTIS Mock AIME 2024-20258.5%2.6%
MATH Level 556.9%39.4%
Omni-MATH27.6%—
LiveBench Math52.3%—
LMArena Math1307—
FrontierMath (Feb 2025 set)2.1%—
FrontierMath Tier 4 (v1)0%—

Knowledge Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 28.6 (#245), Llama 3.2 90B: 21.7 (#274)

BenchmarkClaude 3.5 SonnetLlama 3.2 90B
GPQA Diamond55.3%41%
MMLU87.3%80.3%
Humanity's Last Exam4.1%—
MMLU-Pro77.7%—
Confabulations19.9%—
GPQA (HELM)56.5%—
LMArena Expert1265—

Multimodal Claude 3.5 Sonnet leads

Claude 3.5 Sonnet: 26.5 (#120), Llama 3.2 90B: 25.4 (#124)

BenchmarkClaude 3.5 SonnetLlama 3.2 90B
LMArena Vision11251000
GeoBench62%52%
Video-MME60%—
VPCT33%—

Multilingual Not comparable

Claude 3.5 Sonnet: 43.2 (#185), Llama 3.2 90B: —

BenchmarkClaude 3.5 SonnetLlama 3.2 90B
LMArena Non-English1283—
LMArena Chinese1272—
LMArena French1305—
LMArena German1297—
LMArena Japanese1234—
LMArena Korean1200—
LMArena Russian1306—
LMArena Spanish1290—

Instruction Following Not comparable

Claude 3.5 Sonnet: 68.8 (#182), Llama 3.2 90B: —

BenchmarkClaude 3.5 SonnetLlama 3.2 90B
LiveBench Instruction Following69.3%—
IFEval85.5%—
LMArena Instruction Following1297—

Long Context Not comparable

Claude 3.5 Sonnet: 39.9 (#167), Llama 3.2 90B: —

BenchmarkClaude 3.5 SonnetLlama 3.2 90B
LMArena Longer Query1311—

Writing & Preference Not comparable

Claude 3.5 Sonnet: 52.9 (#164), Llama 3.2 90B: —

BenchmarkClaude 3.5 SonnetLlama 3.2 90B
LMArena Text1298—
LMArena Creative Writing1292—
Short-Story Creative Writing80.3%—
EQ-Bench Creative Writing1451—
WildBench79.2%—
LMArena Multi-Turn1326—
LiveBench Language53.8%—

Frequently asked questions

Is Claude 3.5 Sonnet better than Llama 3.2 90B?

Claude 3.5 Sonnet is the stronger model overall, scoring 34.6 to 27.5 on the Noometry Index.

How many benchmarks do Claude 3.5 Sonnet and Llama 3.2 90B share?

9 benchmarks have published results for both models. Claude 3.5 Sonnet has 60 scored results on Noometry and Llama 3.2 90B has 9.

Related comparisons

Go deeper