Noometry

Model comparison

Claude 3.7 Sonnet vs Grok-2 (Dec 2024)

Claude 3.7 Sonnet is the stronger model overall, scoring 39.4 to 33.4 on the Noometry Index.

Last verified . 32 shared benchmarks.

Claude 3.7 Sonnet Anthropic

39.4

Rank #154 Confirmed

Grok-2 (Dec 2024) xAI

33.4

Rank #233 Confirmed

Summary

  • They share 32 benchmarks with published results for both. Claude 3.7 Sonnet scores higher in 8 categories and Grok-2 (Dec 2024) in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Claude 3.7 Sonnet leads 35.7 to 19.6.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 57.8% for Claude 3.7 Sonnet and 11.5% for Grok-2 (Dec 2024).

Side by side

Claude 3.7 SonnetGrok-2 (Dec 2024)
ProviderAnthropicxAI
Noometry Index39.433.4
Released2025-02-242024-08-13
WeightsProprietaryProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked5834

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 41.1 (#120), Grok-2 (Dec 2024): 32.9 (#254)

BenchmarkClaude 3.7 SonnetGrok-2 (Dec 2024)
LiveBench Coding74.5%46.4%
LMArena Coding13611288
SWE-bench Verified61%—
SWE-bench Verified (bash only)52.8%—
Aider Polyglot64.9%—
GSO3.8%—
WeirdML—22.2%
CadEval54%—

Agentic & Tool Use Not comparable

Claude 3.7 Sonnet: 33.9 (#46), Grok-2 (Dec 2024): —

BenchmarkClaude 3.7 SonnetGrok-2 (Dec 2024)
TheAgentCompany30.9%—
Cybench20%—
DeepResearch Bench43.6%—
OSWorld35.8%—
METR Time Horizons60%—

Reasoning Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 18.7 (#252), Grok-2 (Dec 2024): 16.4 (#294)

BenchmarkClaude 3.7 SonnetGrok-2 (Dec 2024)
SimpleBench46.4%22.7%
LiveBench Reasoning87.8%54.8%
LMArena Hard Prompts13331272
LiveBench Data Analysis74%54.5%
Epoch Capabilities Index141.16130.48
LiveBench76.1%54.3%
ARC-AGI-20.9%—
ARC-AGI-128.6%—
EnigmaEval4.2%—
DTBench—65.2%
ForecastBench61.8—

Math Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 35.7 (#137), Grok-2 (Dec 2024): 19.6 (#278)

BenchmarkClaude 3.7 SonnetGrok-2 (Dec 2024)
OTIS Mock AIME 2024-202557.8%11.5%
LiveBench Math79%54.9%
LMArena Math13361283
MATH Level 591.2%63.5%
FrontierMath (Feb 2025 set)4.1%0.7%
Omni-MATH33%—

Knowledge Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 39.7 (#117), Grok-2 (Dec 2024): 29.8 (#225)

BenchmarkClaude 3.7 SonnetGrok-2 (Dec 2024)
GPQA Diamond79.7%53.8%
Confabulations14.7%20.1%
LMArena Expert13211254
Humanity's Last Exam8%—
MMLU-Pro78.4%—
GPQA (HELM)60.8%—

Multimodal Not comparable

Claude 3.7 Sonnet: 33.1 (#90), Grok-2 (Dec 2024): —

BenchmarkClaude 3.7 SonnetGrok-2 (Dec 2024)
LMArena Vision1169—
GeoBench68%—
VPCT39%—
SpatialViz-Bench33.9%—

Multilingual Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 44.3 (#174), Grok-2 (Dec 2024): 43.3 (#183)

BenchmarkClaude 3.7 SonnetGrok-2 (Dec 2024)
LMArena Non-English12961282
LMArena Chinese12991289
LMArena French13021317
LMArena German13011287
LMArena Japanese12681244
LMArena Korean12491237
LMArena Russian13111286
LMArena Spanish12991281

Instruction Following Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 73.1 (#120), Grok-2 (Dec 2024): 67.1 (#197)

BenchmarkClaude 3.7 SonnetGrok-2 (Dec 2024)
LiveBench Instruction Following81.3%69.6%
LMArena Instruction Following13521270
IFEval83.4%—

Long Context Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 50.2 (#10), Grok-2 (Dec 2024): 38.7 (#186)

BenchmarkClaude 3.7 SonnetGrok-2 (Dec 2024)
LMArena Longer Query13731276
Fiction.LiveBench83.3%—

Writing & Preference Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 54.6 (#145), Grok-2 (Dec 2024): 48.8 (#192)

BenchmarkClaude 3.7 SonnetGrok-2 (Dec 2024)
LMArena Text13141305
LMArena Creative Writing13321284
Short-Story Creative Writing81.1%63.6%
LMArena Multi-Turn13391290
LiveBench Language59.9%45.6%
EQ-Bench Creative Writing1412—
WildBench81.4%—

Frequently asked questions

Is Claude 3.7 Sonnet better than Grok-2 (Dec 2024)?

Claude 3.7 Sonnet is the stronger model overall, scoring 39.4 to 33.4 on the Noometry Index.

Is Claude 3.7 Sonnet or Grok-2 (Dec 2024) better for coding?

Claude 3.7 Sonnet scores higher on coding benchmarks: 41.1 versus 32.9 in the Noometry coding category.

How many benchmarks do Claude 3.7 Sonnet and Grok-2 (Dec 2024) share?

32 benchmarks have published results for both models. Claude 3.7 Sonnet has 58 scored results on Noometry and Grok-2 (Dec 2024) has 34.

Related comparisons

Go deeper