Noometry

Model comparison

Claude 3.7 Sonnet vs Llama 3.2 90B

Claude 3.7 Sonnet is the stronger model overall, scoring 39.4 to 27.1 on the Noometry Index.

Last verified . 7 shared benchmarks.

Claude 3.7 Sonnet Anthropic

39.4

Rank #154 Confirmed

Llama 3.2 90B Meta

27.1

Rank #326 Confirmed

Summary

  • They share 7 benchmarks with published results for both. Claude 3.7 Sonnet scores higher in 4 categories and Llama 3.2 90B in 1 category; 5 gaps are clear of the uncertainty.
  • The widest gap is in math, where Claude 3.7 Sonnet leads 35.7 to 10.4.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 57.8% for Claude 3.7 Sonnet and 2.6% for Llama 3.2 90B.
  • Llama 3.2 90B has downloadable open weights; the other is API-only.

Side by side

Claude 3.7 SonnetLlama 3.2 90B
ProviderAnthropicMeta
Noometry Index39.427.1
Released2025-02-242024-09-24
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked589

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Claude 3.7 Sonnet: 41.1 (#120), Llama 3.2 90B: —

BenchmarkClaude 3.7 SonnetLlama 3.2 90B
SWE-bench Verified61%—
SWE-bench Verified (bash only)52.8%—
Aider Polyglot64.9%—
GSO3.8%—
LiveBench Coding74.5%—
LMArena Coding1361—
CadEval54%—

Agentic & Tool Use Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 33.9 (#46), Llama 3.2 90B: 29.7 (#73)

BenchmarkClaude 3.7 SonnetLlama 3.2 90B
TheAgentCompany30.9%—
Cybench20%—
DeepResearch Bench43.6%—
OSWorld35.8%—
BALROG—27.3%
METR Time Horizons60%—

Reasoning Llama 3.2 90B leads

Claude 3.7 Sonnet: 18.7 (#252), Llama 3.2 90B: 20.9 (#207)

BenchmarkClaude 3.7 SonnetLlama 3.2 90B
EnigmaEval4.2%0.4%
Epoch Capabilities Index141.16125.5
ARC-AGI-20.9%—
SimpleBench46.4%—
ARC-AGI-128.6%—
LiveBench Reasoning87.8%—
LMArena Hard Prompts1333—
LiveBench Data Analysis74%—
ForecastBench61.8—
LiveBench76.1%—

Math Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 35.7 (#137), Llama 3.2 90B: 10.4 (#302)

BenchmarkClaude 3.7 SonnetLlama 3.2 90B
OTIS Mock AIME 2024-202557.8%2.6%
MATH Level 591.2%39.4%
Omni-MATH33%—
LiveBench Math79%—
LMArena Math1336—
FrontierMath (Feb 2025 set)4.1%—

Knowledge Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 39.7 (#117), Llama 3.2 90B: 21.7 (#269)

BenchmarkClaude 3.7 SonnetLlama 3.2 90B
GPQA Diamond79.7%41%
Humanity's Last Exam8%—
MMLU-Pro78.4%—
Confabulations14.7%—
GPQA (HELM)60.8%—
LMArena Expert1321—
MMLU—80.3%

Multimodal Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 33.1 (#90), Llama 3.2 90B: 24.9 (#119)

BenchmarkClaude 3.7 SonnetLlama 3.2 90B
LMArena Vision11691000
GeoBench68%52%
VPCT39%—
SpatialViz-Bench33.9%—

Multilingual Not comparable

Claude 3.7 Sonnet: 44.3 (#174), Llama 3.2 90B: —

BenchmarkClaude 3.7 SonnetLlama 3.2 90B
LMArena Non-English1296—
LMArena Chinese1299—
LMArena French1302—
LMArena German1301—
LMArena Japanese1268—
LMArena Korean1249—
LMArena Russian1311—
LMArena Spanish1299—

Instruction Following Not comparable

Claude 3.7 Sonnet: 73.1 (#120), Llama 3.2 90B: —

BenchmarkClaude 3.7 SonnetLlama 3.2 90B
LiveBench Instruction Following81.3%—
IFEval83.4%—
LMArena Instruction Following1352—

Long Context Not comparable

Claude 3.7 Sonnet: 50.2 (#10), Llama 3.2 90B: —

BenchmarkClaude 3.7 SonnetLlama 3.2 90B
Fiction.LiveBench83.3%—
LMArena Longer Query1373—

Writing & Preference Not comparable

Claude 3.7 Sonnet: 54.6 (#145), Llama 3.2 90B: —

BenchmarkClaude 3.7 SonnetLlama 3.2 90B
LMArena Text1314—
LMArena Creative Writing1332—
Short-Story Creative Writing81.1%—
EQ-Bench Creative Writing1412—
WildBench81.4%—
LMArena Multi-Turn1339—
LiveBench Language59.9%—

Frequently asked questions

Is Claude 3.7 Sonnet better than Llama 3.2 90B?

Claude 3.7 Sonnet is the stronger model overall, scoring 39.4 to 27.1 on the Noometry Index.

How many benchmarks do Claude 3.7 Sonnet and Llama 3.2 90B share?

7 benchmarks have published results for both models. Claude 3.7 Sonnet has 58 scored results on Noometry and Llama 3.2 90B has 9.

Related comparisons

Go deeper