Noometry

Model comparison

Claude Sonnet 4 vs GPT-5.6 Sol

GPT-5.6 Sol is the stronger model overall, scoring 65.0 to 40.8 on the Noometry Index.

Last verified . 35 shared benchmarks.

Claude Sonnet 4 Anthropic

40.8

Rank #145 Confirmed

GPT-5.6 Sol OpenAI

65.0

Rank #7 Confirmed

Summary

  • They share 35 benchmarks with published results for both. Claude Sonnet 4 scores higher in 0 categories and GPT-5.6 Sol in 10 categories; 10 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where GPT-5.6 Sol leads 74.8 to 22.9.
  • The biggest single-benchmark swing is ARC-AGI-2: 5.9% for Claude Sonnet 4 and 92.5% for GPT-5.6 Sol.
  • Claude Sonnet 4 is cheaper at $3 / $15 per million input/output tokens, against $4 / $20 for GPT-5.6 Sol.
  • GPT-5.6 Sol accepts more context: 1.05M tokens versus 200K.

Side by side

Claude Sonnet 4GPT-5.6 Sol
ProviderAnthropicOpenAI
Noometry Index40.865.0
Released2025-05-222026-07-09
WeightsProprietaryProprietary
Context window200K1.05M
Max output64K128K
Input $ / M tokens$3$4
Output $ / M tokens$15$20
Results tracked5865

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.6 Sol leads

Claude Sonnet 4: 43.5 (#88), GPT-5.6 Sol: 65.1 (#7)

BenchmarkClaude Sonnet 4GPT-5.6 Sol
SciCode40%57.1%
GSO4.9%76.5%
WeirdML46.1%89.4%
LMArena Coding14141498
ALE-Bench655.352,177
DeepSWE—72.7%
FrontierCode—47.5%
SWE-bench Verified (bash only)64.9%—
Aider Polyglot61.3%—
CursorBench—41.7%
LMArena WebDev—1618
FrontierSWE—32.2%
MirrorCode—20%

Agentic & Tool Use GPT-5.6 Sol leads

Claude Sonnet 4: 38.5 (#31), GPT-5.6 Sol: 50.3 (#7)

BenchmarkClaude Sonnet 4GPT-5.6 Sol
APEX-Agents—51.4%
OSWorld 2.0—27.3%
TheAgentCompany33.1%—
τ²-bench Banking—46.9%
Cybench35%—
DeepResearch Bench46.6%—
OSWorld43.9%—
PostTrainBench—36.2%
BALROG—60%
GBAEval—52.6%
GDP.pdf—30.7%
LMArena Search—1257
METR Time Horizons62%—
Vending-Bench 2—9,619

Reasoning GPT-5.6 Sol leads

Claude Sonnet 4: 22.9 (#187), GPT-5.6 Sol: 74.8 (#8)

BenchmarkClaude Sonnet 4GPT-5.6 Sol
ARC-AGI-25.9%92.5%
SimpleBench45.5%71.7%
Kagi LLM Benchmark73%67%
ARC-AGI-140%97.5%
CritPt0.3%32.3%
EnigmaEval3.1%37.1%
LMArena Hard Prompts13721484
DTBench77.1%96%
LMCA29%59.2%
Epoch Capabilities Index141.69161.66
NYT Connections (extended)—93.8%
Chess Puzzles—64%
EBR-Bench—44.8%
Mystery Game Puzzles—58%
Surface Evolver Bench—93.1%
Bench to the Future 3—0.14
ForecastBench60.2—

Math GPT-5.6 Sol leads

Claude Sonnet 4: 43.3 (#80), GPT-5.6 Sol: 85.6 (#9)

BenchmarkClaude Sonnet 4GPT-5.6 Sol
OTIS Mock AIME 2024-202571.1%100%
LMArena Math13751474
FrontierMath (Tiers 1-3)—89.1%
FrontierMath Tier 4—82.9%
ProofBench—83%
Omni-MATH60.2%—
MATH Level 584.4%—
FrontierMath (Feb 2025 set)4.1%—
FrontierMath Erdős—0%
FrontierMath Tier 4 (v1)0%—

Knowledge GPT-5.6 Sol leads

Claude Sonnet 4: 41.8 (#108), GPT-5.6 Sol: 64.3 (#18)

BenchmarkClaude Sonnet 4GPT-5.6 Sol
GPQA Diamond79.2%93.5%
Vectara Hallucination Rate10.3%12.4%
LMArena Expert13721516
Humanity's Last Exam7.8%—
SimpleQA Verified—69.7%
MMLU-Pro84.3%—
Confabulations13.2%—
GPQA (HELM)70.6%—

Multimodal GPT-5.6 Sol leads

Claude Sonnet 4: 26.2 (#121), GPT-5.6 Sol: 48.6 (#9)

BenchmarkClaude Sonnet 4GPT-5.6 Sol
LMArena Vision11911281
GeoBench37%—
VPCT34%—
Blueprint-Bench 2—33.6%
Furniture Assembly—56.7%
LMArena Document—1483
MindCube44.8%—

Multilingual GPT-5.6 Sol leads

Claude Sonnet 4: 46.7 (#156), GPT-5.6 Sol: 55.3 (#32)

BenchmarkClaude Sonnet 4GPT-5.6 Sol
LMArena Non-English13331452
LMArena Chinese13501527
LMArena French13631477
LMArena German13311476
LMArena Japanese13021471
LMArena Korean12911442
LMArena Russian13551468
LMArena Spanish13571441

Instruction Following GPT-5.6 Sol leads

Claude Sonnet 4: 71.7 (#145), GPT-5.6 Sol: 77.7 (#16)

BenchmarkClaude Sonnet 4GPT-5.6 Sol
LMArena Instruction Following13761482
IFEval84%—

Long Context GPT-5.6 Sol leads

Claude Sonnet 4: 33.7 (#259), GPT-5.6 Sol: 45.4 (#42)

BenchmarkClaude Sonnet 4GPT-5.6 Sol
LMArena Longer Query13981480
Fiction.LiveBench46.9%—

Writing & Preference GPT-5.6 Sol leads

Claude Sonnet 4: 57.1 (#132), GPT-5.6 Sol: 73.3 (#12)

BenchmarkClaude Sonnet 4GPT-5.6 Sol
LMArena Text13511457
LMArena Creative Writing13451448
EQ-Bench Creative Writing14831972
LMArena Multi-Turn13761460
Short-Story Creative Writing81.4%—
WildBench83.8%—
EQ-Bench 4—1250

Frequently asked questions

Is Claude Sonnet 4 better than GPT-5.6 Sol?

GPT-5.6 Sol is the stronger model overall, scoring 65.0 to 40.8 on the Noometry Index.

Which is cheaper, Claude Sonnet 4 or GPT-5.6 Sol?

Claude Sonnet 4 is cheaper. It lists at $3 per million input tokens and $15 per million output tokens; GPT-5.6 Sol lists at $4 and $20.

Is Claude Sonnet 4 or GPT-5.6 Sol better for coding?

GPT-5.6 Sol scores higher on coding benchmarks: 65.1 versus 43.5 in the Noometry coding category.

Which has the bigger context window?

GPT-5.6 Sol does, with 1.05M tokens against 200K.

How many benchmarks do Claude Sonnet 4 and GPT-5.6 Sol share?

35 benchmarks have published results for both models. Claude Sonnet 4 has 58 scored results on Noometry and GPT-5.6 Sol has 65.

Related comparisons

Go deeper