Noometry

Model comparison

Claude 2.1 vs GPT-5.4

GPT-5.4 is the stronger model overall, scoring 59.4 to 25.2 on the Noometry Index.

Last verified . 6 shared benchmarks.

Claude 2.1 Anthropic

25.2

Rank #345 Reported

GPT-5.4 OpenAI

59.4

Rank #16 Confirmed

Summary

  • They share 6 benchmarks with published results for both. Claude 2.1 scores higher in 0 categories and GPT-5.4 in 4 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in math, where GPT-5.4 leads 73.5 to 10.2.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 1.9% for Claude 2.1 and 97.8% for GPT-5.4.

Side by side

Claude 2.1GPT-5.4
ProviderAnthropicOpenAI
Noometry Index25.259.4
Released2023-11-212026-03-05
WeightsProprietaryProprietary
Context window—1.05M
Max output—128K
Input $ / M tokens—$2.50
Output $ / M tokens—$15
Results tracked768

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.4 leads

Claude 2.1: 26.2 (#327), GPT-5.4: 52.6 (#33)

BenchmarkClaude 2.1GPT-5.4
WeirdML7.1%77.7%
SWE-bench Verified—76.9%
DeepSWE—51.8%
LMArena WebDev—1465
SciCode—56.6%
GSO—31.4%
LMArena Coding—1497
MirrorCode—15.6%
ALE-Bench—1,607
AlgoTune—1.85

Agentic & Tool Use Not comparable

Claude 2.1: —, GPT-5.4: 46.5 (#13)

BenchmarkClaude 2.1GPT-5.4
Terminal-Bench—81.8%
APEX-Agents—52.4%
τ²-bench Banking—39.4%
DeepResearch Bench—35.1%
PostTrainBench—19%
GBAEval—45.1%
LMArena Search—1197
METR Time Horizons—74.3%
Vending-Bench 2—6,144

Reasoning GPT-5.4 leads

Claude 2.1: 21.4 (#221), GPT-5.4: 61.8 (#19)

BenchmarkClaude 2.1GPT-5.4
DTBench51%94.4%
Epoch Capabilities Index119.27156.81
ForecastBench54.259.5
ARC-AGI-2—74%
Kagi LLM Benchmark—63.8%
NYT Connections (extended)—91.3%
ARC-AGI-1—93.7%
CritPt—23.4%
Chess Puzzles—44%
EnigmaEval—16%
Thematic Generalization—80%
EBR-Bench—25.4%
LMArena Hard Prompts—1485
Mystery Game Puzzles—37%
LMCA—52%

Math GPT-5.4 leads

Claude 2.1: 10.2 (#315), GPT-5.4: 73.5 (#19)

BenchmarkClaude 2.1GPT-5.4
OTIS Mock AIME 2024-20251.9%97.8%
FrontierMath (Tiers 1-3)—78.6%
FrontierMath Tier 4—49%
MathArena Final-Answer Competitions—83.1%
ProofBench—56%
LMArena Math—1488
FrontierMath (Feb 2025 set)—47.6%
FrontierMath Tier 4 (v1)—27.1%

Knowledge GPT-5.4 leads

Claude 2.1: 15.4 (#292), GPT-5.4: 65.3 (#14)

BenchmarkClaude 2.1GPT-5.4
GPQA Diamond33%93.3%
Humanity's Last Exam—36.2%
SimpleQA Verified—45.1%
Vectara Hallucination Rate—7%
LMArena Expert—1507
MMLU73.5%—

Multimodal Not comparable

Claude 2.1: —, GPT-5.4: 43.7 (#20)

BenchmarkClaude 2.1GPT-5.4
LMArena Vision—1303
Blueprint-Bench 2—27.1%
Furniture Assembly—37.5%
LMArena Document—1471

Multilingual Not comparable

Claude 2.1: —, GPT-5.4: 56.2 (#23)

BenchmarkClaude 2.1GPT-5.4
LMArena Non-English—1465
LMArena Chinese—1519
LMArena French—1493
LMArena German—1472
LMArena Japanese—1485
LMArena Korean—1448
LMArena Russian—1480
LMArena Spanish—1454

Instruction Following Not comparable

Claude 2.1: —, GPT-5.4: 77.1 (#27)

BenchmarkClaude 2.1GPT-5.4
LMArena Instruction Following—1469

Long Context Not comparable

Claude 2.1: —, GPT-5.4: 50.3 (#8)

BenchmarkClaude 2.1GPT-5.4
CL-bench—27.9%
CL-bench Life—21.7%
LMArena Longer Query—1473

Writing & Preference Not comparable

Claude 2.1: —, GPT-5.4: 71.9 (#17)

BenchmarkClaude 2.1GPT-5.4
LMArena Text—1469
LMArena Creative Writing—1439
EQ-Bench Creative Writing—1840
EQ-Bench 4—1272
LMArena Multi-Turn—1482

Frequently asked questions

Is Claude 2.1 better than GPT-5.4?

GPT-5.4 is the stronger model overall, scoring 59.4 to 25.2 on the Noometry Index.

Is Claude 2.1 or GPT-5.4 better for coding?

GPT-5.4 scores higher on coding benchmarks: 52.6 versus 26.2 in the Noometry coding category.

How many benchmarks do Claude 2.1 and GPT-5.4 share?

6 benchmarks have published results for both models. Claude 2.1 has 7 scored results on Noometry and GPT-5.4 has 68.

Related comparisons

Go deeper