Noometry

Model comparison

Claude 2.1 vs GPT-5.6 Sol

GPT-5.6 Sol is the stronger model overall, scoring 64.4 to 24.9 on the Noometry Index.

Last verified . 5 shared benchmarks.

Claude 2.1 Anthropic

24.9

Rank #340 Reported

GPT-5.6 Sol OpenAI

64.4

Rank #6 Confirmed

Summary

  • They share 5 benchmarks with published results for both. Claude 2.1 scores higher in 0 categories and GPT-5.6 Sol in 4 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in math, where GPT-5.6 Sol leads 85.5 to 9.6.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 1.9% for Claude 2.1 and 100% for GPT-5.6 Sol.

Side by side

Claude 2.1GPT-5.6 Sol
ProviderAnthropicOpenAI
Noometry Index24.964.4
Released2023-11-212026-07-09
WeightsProprietaryProprietary
Context window—1.05M
Max output—128K
Input $ / M tokens—$4
Output $ / M tokens—$20
Results tracked764

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.6 Sol leads

Claude 2.1: 25.8 (#322), GPT-5.6 Sol: 61.1 (#7)

BenchmarkClaude 2.1GPT-5.6 Sol
WeirdML7.1%89.4%
DeepSWE—72.7%
FrontierCode—47.5%
CursorBench—41.7%
LMArena WebDev—1619
FrontierSWE—32.2%
SciCode—57.1%
LMArena Coding—1493
MirrorCode—20%
ALE-Bench—2,177

Agentic & Tool Use Not comparable

Claude 2.1: —, GPT-5.6 Sol: 50.2 (#7)

BenchmarkClaude 2.1GPT-5.6 Sol
APEX-Agents—51.4%
OSWorld 2.0—27.3%
τ²-bench Banking—46.9%
PostTrainBench—36.2%
BALROG—60%
GBAEval—52.6%
GDP.pdf—30.7%
LMArena Search—1257
Vending-Bench 2—9,619

Reasoning GPT-5.6 Sol leads

Claude 2.1: 20.6 (#216), GPT-5.6 Sol: 74.7 (#6)

BenchmarkClaude 2.1GPT-5.6 Sol
DTBench51%96%
Epoch Capabilities Index119.22161.99
ARC-AGI-2—92.5%
SimpleBench—71.7%
Kagi LLM Benchmark—67%
NYT Connections (extended)—93.8%
ARC-AGI-1—97.5%
CritPt—32.3%
Chess Puzzles—64%
EnigmaEval—37.1%
EBR-Bench—44.8%
LMArena Hard Prompts—1481
Mystery Game Puzzles—58%
LMCA—59.2%
Surface Evolver Bench—93.1%
Bench to the Future 3—0.14
ForecastBench54.2—

Math GPT-5.6 Sol leads

Claude 2.1: 9.6 (#309), GPT-5.6 Sol: 85.5 (#7)

BenchmarkClaude 2.1GPT-5.6 Sol
OTIS Mock AIME 2024-20251.9%100%
FrontierMath (Tiers 1-3)—89.1%
FrontierMath Tier 4—82.9%
ProofBench—83%
LMArena Math—1472
FrontierMath Erdős—0%

Knowledge GPT-5.6 Sol leads

Claude 2.1: 15.5 (#287), GPT-5.6 Sol: 64.3 (#16)

BenchmarkClaude 2.1GPT-5.6 Sol
GPQA Diamond33%93.5%
SimpleQA Verified—69.7%
Vectara Hallucination Rate—12.4%
LMArena Expert—1517
MMLU73.5%—

Multimodal Not comparable

Claude 2.1: —, GPT-5.6 Sol: 48.9 (#8)

BenchmarkClaude 2.1GPT-5.6 Sol
LMArena Vision—1281
Blueprint-Bench 2—33.6%
Furniture Assembly—56.7%
LMArena Document—1483

Multilingual Not comparable

Claude 2.1: —, GPT-5.6 Sol: 55.4 (#32)

BenchmarkClaude 2.1GPT-5.6 Sol
LMArena Non-English—1450
LMArena Chinese—1523
LMArena French—1475
LMArena German—1476
LMArena Japanese—1475
LMArena Korean—1434
LMArena Russian—1466
LMArena Spanish—1449

Instruction Following Not comparable

Claude 2.1: —, GPT-5.6 Sol: 77.8 (#16)

BenchmarkClaude 2.1GPT-5.6 Sol
LMArena Instruction Following—1480

Long Context Not comparable

Claude 2.1: —, GPT-5.6 Sol: 45.3 (#42)

BenchmarkClaude 2.1GPT-5.6 Sol
LMArena Longer Query—1479

Writing & Preference Not comparable

Claude 2.1: —, GPT-5.6 Sol: 73.5 (#12)

BenchmarkClaude 2.1GPT-5.6 Sol
LMArena Text—1456
LMArena Creative Writing—1450
EQ-Bench Creative Writing—1972
EQ-Bench 4—1250
LMArena Multi-Turn—1456

Frequently asked questions

Is Claude 2.1 better than GPT-5.6 Sol?

GPT-5.6 Sol is the stronger model overall, scoring 64.4 to 24.9 on the Noometry Index.

Is Claude 2.1 or GPT-5.6 Sol better for coding?

GPT-5.6 Sol scores higher on coding benchmarks: 61.1 versus 25.8 in the Noometry coding category.

How many benchmarks do Claude 2.1 and GPT-5.6 Sol share?

5 benchmarks have published results for both models. Claude 2.1 has 7 scored results on Noometry and GPT-5.6 Sol has 64.

Related comparisons

Go deeper