Noometry

Model comparison

Claude 3.7 Sonnet vs Mercury

Claude 3.7 Sonnet is the stronger model overall, scoring 39.5 to 37.6 on the Noometry Index.

Last verified . 8 shared benchmarks.

Claude 3.7 Sonnet Anthropic

39.5

Rank #164 Confirmed

Mercury Inception

37.6

Rank #199 Confirmed

Summary

  • They share 8 benchmarks with published results for both. Claude 3.7 Sonnet scores higher in 6 categories and Mercury in 0 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in long context, where Claude 3.7 Sonnet leads 50.3 to 38.4.

Side by side

Claude 3.7 SonnetMercury
ProviderAnthropicInception
Noometry Index39.537.6
Released2025-02-24—
WeightsProprietaryProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked589

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 40.6 (#136), Mercury: 38.7 (#170)

BenchmarkClaude 3.7 SonnetMercury
LMArena Coding13611322
SWE-bench Verified61%—
SWE-bench Verified (bash only)52.8%—
Aider Polyglot64.9%—
GSO3.8%—
LiveBench Coding74.5%—
CadEval54%—

Agentic & Tool Use Not comparable

Claude 3.7 Sonnet: 34.1 (#50), Mercury: —

BenchmarkClaude 3.7 SonnetMercury
TheAgentCompany30.9%—
Cybench20%—
DeepResearch Bench43.6%—
OSWorld35.8%—
METR Time Horizons60%—

Reasoning Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 18.6 (#277), Mercury: 17.5 (#293)

BenchmarkClaude 3.7 SonnetMercury
LMArena Hard Prompts13331285
ARC-AGI-20.9%—
SimpleBench46.4%—
Kagi LLM Benchmark—21.6%
ARC-AGI-128.6%—
EnigmaEval4.2%—
LiveBench Reasoning87.8%—
LiveBench Data Analysis74%—
Epoch Capabilities Index141.16—
ForecastBench61.8—
LiveBench76.1%—

Math Not comparable

Claude 3.7 Sonnet: 37.5 (#153), Mercury: —

BenchmarkClaude 3.7 SonnetMercury
OTIS Mock AIME 2024-202557.8%—
Omni-MATH33%—
LiveBench Math79%—
LMArena Math1337—
MATH Level 591.2%—
FrontierMath (Feb 2025 set)4.1%—

Knowledge Not comparable

Claude 3.7 Sonnet: 39.8 (#130), Mercury: —

BenchmarkClaude 3.7 SonnetMercury
GPQA Diamond79.7%—
Humanity's Last Exam8%—
MMLU-Pro78.4%—
Confabulations14.7%—
GPQA (HELM)60.8%—
LMArena Expert1321—

Multimodal Not comparable

Claude 3.7 Sonnet: 33.7 (#95), Mercury: —

BenchmarkClaude 3.7 SonnetMercury
LMArena Vision1169—
GeoBench68%—
VPCT39%—
SpatialViz-Bench33.9%—

Multilingual Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 44.1 (#179), Mercury: 41.6 (#206)

BenchmarkClaude 3.7 SonnetMercury
LMArena Non-English12961260
LMArena Chinese1299—
LMArena French1303—
LMArena German1301—
LMArena Japanese1267—
LMArena Korean1249—
LMArena Russian1311—
LMArena Spanish1298—

Instruction Following Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 72.9 (#125), Mercury: 65.2 (#224)

BenchmarkClaude 3.7 SonnetMercury
LMArena Instruction Following13521239
LiveBench Instruction Following81.3%—
IFEval83.4%—

Long Context Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 50.3 (#10), Mercury: 38.4 (#198)

BenchmarkClaude 3.7 SonnetMercury
LMArena Longer Query13731266
Fiction.LiveBench83.3%—

Writing & Preference Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 54.4 (#150), Mercury: 46.2 (#221)

BenchmarkClaude 3.7 SonnetMercury
LMArena Text13141282
LMArena Creative Writing13321191
LMArena Multi-Turn13391282
Short-Story Creative Writing81.1%—
EQ-Bench Creative Writing1412—
WildBench81.4%—
LiveBench Language59.9%—

Frequently asked questions

Is Claude 3.7 Sonnet better than Mercury?

Claude 3.7 Sonnet is the stronger model overall, scoring 39.5 to 37.6 on the Noometry Index.

Is Claude 3.7 Sonnet or Mercury better for coding?

Claude 3.7 Sonnet scores higher on coding benchmarks: 40.6 versus 38.7 in the Noometry coding category.

How many benchmarks do Claude 3.7 Sonnet and Mercury share?

8 benchmarks have published results for both models. Claude 3.7 Sonnet has 58 scored results on Noometry and Mercury has 9.

Related comparisons

Go deeper