Noometry

Model comparison

Claude 3 Sonnet vs GPT-5.4 nano

GPT-5.4 nano is the stronger model overall, scoring 41.9 to 28.8 on the Noometry Index.

Last verified . 23 shared benchmarks.

Claude 3 Sonnet Anthropic

28.8

Rank #311 Confirmed

GPT-5.4 nano OpenAI

41.9

Rank #111 Confirmed

Summary

  • They share 23 benchmarks with published results for both. Claude 3 Sonnet scores higher in 0 categories and GPT-5.4 nano in 9 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in math, where GPT-5.4 nano leads 40.7 to 10.0.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 2.5% for Claude 3 Sonnet and 87.8% for GPT-5.4 nano.

Side by side

Claude 3 SonnetGPT-5.4 nano
ProviderAnthropicOpenAI
Noometry Index28.841.9
Released2024-02-292026-03-17
WeightsProprietaryProprietary
Context window—400K
Max output—128K
Input $ / M tokens—$0.20
Output $ / M tokens—$1.25
Results tracked3040

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.4 nano leads

Claude 3 Sonnet: 29.2 (#296), GPT-5.4 nano: 43.1 (#82)

BenchmarkClaude 3 SonnetGPT-5.4 nano
WeirdML10.2%49.2%
LMArena Coding12231404
SciCode—46.9%
BigCodeBench Instruct42.7%—
BigCodeBench Complete53.8%—
ALE-Bench—1,005
HumanEval+64%—
MBPP+69.3%—

Reasoning GPT-5.4 nano leads

Claude 3 Sonnet: 19.7 (#232), GPT-5.4 nano: 23.8 (#159)

BenchmarkClaude 3 SonnetGPT-5.4 nano
LMArena Hard Prompts11971380
DTBench53.6%80.3%
Epoch Capabilities Index120.69145.78
ARC-AGI-2—5.7%
Kagi LLM Benchmark—39.7%
ARC-AGI-1—51.5%
CritPt—9.3%
Chess Puzzles—30%
Mystery Game Puzzles—9%
LMCA—36.9%
ForecastBench—57.3
WinoGrande75.1%—

Math GPT-5.4 nano leads

Claude 3 Sonnet: 10.0 (#305), GPT-5.4 nano: 40.7 (#76)

BenchmarkClaude 3 SonnetGPT-5.4 nano
OTIS Mock AIME 2024-20252.5%87.8%
LMArena Math12131406
FrontierMath (Tiers 1-3)—44.9%
FrontierMath Tier 4—12.2%
ProofBench—5%
MATH Level 518.2%—
FrontierMath (Feb 2025 set)—25.9%
FrontierMath Tier 4 (v1)—6.3%

Knowledge GPT-5.4 nano leads

Claude 3 Sonnet: 21.1 (#271), GPT-5.4 nano: 41.8 (#94)

BenchmarkClaude 3 SonnetGPT-5.4 nano
GPQA Diamond40.6%78.5%
LMArena Expert11731397
SimpleQA Verified—11.7%
Vectara Hallucination Rate—3.1%
MMLU75.9%—

Multimodal GPT-5.4 nano leads

Claude 3 Sonnet: 24.7 (#120), GPT-5.4 nano: 36.0 (#73)

BenchmarkClaude 3 SonnetGPT-5.4 nano
LMArena Vision9841196

Multilingual GPT-5.4 nano leads

Claude 3 Sonnet: 38.0 (#229), GPT-5.4 nano: 48.8 (#136)

BenchmarkClaude 3 SonnetGPT-5.4 nano
LMArena Non-English12051359
LMArena Chinese11881391
LMArena French12281396
LMArena German12041368
LMArena Japanese11311342
LMArena Korean11281321
LMArena Russian12271363
LMArena Spanish12041368

Instruction Following GPT-5.4 nano leads

Claude 3 Sonnet: 63.1 (#230), GPT-5.4 nano: 72.1 (#139)

BenchmarkClaude 3 SonnetGPT-5.4 nano
LMArena Instruction Following12001362

Long Context GPT-5.4 nano leads

Claude 3 Sonnet: 36.7 (#223), GPT-5.4 nano: 41.6 (#131)

BenchmarkClaude 3 SonnetGPT-5.4 nano
LMArena Longer Query12111365

Writing & Preference GPT-5.4 nano leads

Claude 3 Sonnet: 42.3 (#233), GPT-5.4 nano: 55.9 (#137)

BenchmarkClaude 3 SonnetGPT-5.4 nano
LMArena Text12181372
LMArena Creative Writing11861313
LMArena Multi-Turn12271382

Frequently asked questions

Is Claude 3 Sonnet better than GPT-5.4 nano?

GPT-5.4 nano is the stronger model overall, scoring 41.9 to 28.8 on the Noometry Index.

Is Claude 3 Sonnet or GPT-5.4 nano better for coding?

GPT-5.4 nano scores higher on coding benchmarks: 43.1 versus 29.2 in the Noometry coding category.

How many benchmarks do Claude 3 Sonnet and GPT-5.4 nano share?

23 benchmarks have published results for both models. Claude 3 Sonnet has 30 scored results on Noometry and GPT-5.4 nano has 40.

Related comparisons

Go deeper