Noometry

Model comparison

Claude Sonnet 5.5 vs Llama 3.1-405B

Claude Sonnet 5.5 is the stronger model overall, scoring 61.9 to 30.7 on the Noometry Index.

Last verified . 15 shared benchmarks.

Claude Sonnet 5.5 Anthropic

61.9

Rank #10 Confirmed

Llama 3.1-405B Meta

30.7

Rank #288 Confirmed

Summary

  • They share 15 benchmarks with published results for both. Claude Sonnet 5.5 scores higher in 9 categories and Llama 3.1-405B in 0 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in math, where Claude Sonnet 5.5 leads 87.9 to 18.4.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 100% for Claude Sonnet 5.5 and 9.7% for Llama 3.1-405B.
  • Llama 3.1-405B has downloadable open weights; the other is API-only.

Side by side

Claude Sonnet 5.5Llama 3.1-405B
ProviderAnthropicMeta
Noometry Index61.930.7
Released2026-09-282024-07-23
WeightsProprietaryOpen
Context window1M—
Max output128K—
Input $ / M tokens$2—
Output $ / M tokens$10—
Results tracked3242

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude Sonnet 5.5 leads

Claude Sonnet 5.5: 67.3 (#6), Llama 3.1-405B: 33.1 (#262)

BenchmarkClaude Sonnet 5.5Llama 3.1-405B
LMArena Coding15131291
FrontierCode52.1%—
CursorBench55.5%—
LMArena WebDev1774—
FrontierSWE61.9%—
SciCode61%—
WeirdML—21.4%
ALE-Bench1,819—

Agentic & Tool Use Claude Sonnet 5.5 leads

Claude Sonnet 5.5: 45.0 (#16), Llama 3.1-405B: 21.0 (#140)

BenchmarkClaude Sonnet 5.5Llama 3.1-405B
APEX-Agents75.5%—
TheAgentCompany—7.4%
Cybench—7.5%

Reasoning Claude Sonnet 5.5 leads

Claude Sonnet 5.5: 54.0 (#28), Llama 3.1-405B: 16.8 (#300)

BenchmarkClaude Sonnet 5.5Llama 3.1-405B
LMArena Hard Prompts14951269
Epoch Capabilities Index165.03128.75
SimpleBench—23%
Kagi LLM Benchmark—45%
NYT Connections (extended)80.5%—
CritPt31.4%—
Mystery Game Puzzles65%—
DTBench—61.4%
BIG-Bench Hard—82.9%
ForecastBench—59.9
HellaSwag—89.2%
PIQA—85.9%
WinoGrande—89.2%

Math Claude Sonnet 5.5 leads

Claude Sonnet 5.5: 87.9 (#6), Llama 3.1-405B: 18.4 (#290)

BenchmarkClaude Sonnet 5.5Llama 3.1-405B
OTIS Mock AIME 2024-2025100%9.7%
LMArena Math15101281
FrontierMath (Tiers 1-3)88.8%—
FrontierMath Tier 480.5%—
ProofBench100%—
Omni-MATH—24.9%
MATH Level 5—49.8%
FrontierMath Erdős2.9%—

Knowledge Claude Sonnet 5.5 leads

Claude Sonnet 5.5: 66.0 (#12), Llama 3.1-405B: 30.4 (#227)

BenchmarkClaude Sonnet 5.5Llama 3.1-405B
GPQA Diamond95.6%50.9%
LMArena Expert15401243
SimpleQA Verified46.5%—
MMLU-Pro—72.3%
Confabulations—17.6%
GPQA (HELM)—52.2%
ARC (AI2) Challenge—95.3%
MMLU—84.5%
TriviaQA—82.7%

Multimodal Not comparable

Claude Sonnet 5.5: 51.5 (#6), Llama 3.1-405B: —

BenchmarkClaude Sonnet 5.5Llama 3.1-405B
LMArena Vision1289—
Furniture Assembly75%—

Multilingual Claude Sonnet 5.5 leads

Claude Sonnet 5.5: 55.3 (#30), Llama 3.1-405B: 40.7 (#214)

BenchmarkClaude Sonnet 5.5Llama 3.1-405B
LMArena Non-English14521248
LMArena Chinese15221242
LMArena Russian14511265
LMArena French—1279
LMArena German—1252
LMArena Japanese—1208
LMArena Korean—1184
LMArena Spanish—1260

Instruction Following Claude Sonnet 5.5 leads

Claude Sonnet 5.5: 78.3 (#11), Llama 3.1-405B: 65.9 (#214)

BenchmarkClaude Sonnet 5.5Llama 3.1-405B
LMArena Instruction Following14951259
IFEval—81.1%

Long Context Claude Sonnet 5.5 leads

Claude Sonnet 5.5: 45.9 (#28), Llama 3.1-405B: 38.4 (#197)

BenchmarkClaude Sonnet 5.5Llama 3.1-405B
LMArena Longer Query14981266

Writing & Preference Claude Sonnet 5.5 leads

Claude Sonnet 5.5: 66.0 (#40), Llama 3.1-405B: 38.9 (#251)

BenchmarkClaude Sonnet 5.5Llama 3.1-405B
LMArena Text14711284
LMArena Creative Writing14651262
LMArena Multi-Turn14741297
EQ-Bench Creative Writing—870
WildBench—78.3%

Frequently asked questions

Is Claude Sonnet 5.5 better than Llama 3.1-405B?

Claude Sonnet 5.5 is the stronger model overall, scoring 61.9 to 30.7 on the Noometry Index.

Is Claude Sonnet 5.5 or Llama 3.1-405B better for coding?

Claude Sonnet 5.5 scores higher on coding benchmarks: 67.3 versus 33.1 in the Noometry coding category.

How many benchmarks do Claude Sonnet 5.5 and Llama 3.1-405B share?

15 benchmarks have published results for both models. Claude Sonnet 5.5 has 32 scored results on Noometry and Llama 3.1-405B has 42.

Related comparisons

Go deeper