Noometry

Model comparison

Claude 3 Opus vs GPT-4 Turbo

GPT-4 Turbo is the stronger model overall, scoring 30.3 to 29.3 on the Noometry Index.

Last verified . 35 shared benchmarks.

Claude 3 Opus Anthropic

29.3

Rank #300 Confirmed

GPT-4 Turbo OpenAI

30.3

Rank #281 Confirmed

Summary

  • They share 35 benchmarks with published results for both. Claude 3 Opus scores higher in 4 categories and GPT-4 Turbo in 5 categories; 3 gaps are clear of the uncertainty.
  • The widest gap is in math, where Claude 3 Opus leads 13.9 to 8.6.
  • The biggest single-benchmark swing is MATH Level 5: 37.5% for Claude 3 Opus and 46.7% for GPT-4 Turbo.

Side by side

Claude 3 OpusGPT-4 Turbo
ProviderAnthropicOpenAI
Noometry Index29.330.3
Released2024-02-292023-11-06
WeightsProprietaryProprietary
Context window—128K
Max output—4K
Input $ / M tokens—$10
Output $ / M tokens—$30
Results tracked4636

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

Claude 3 Opus: 32.4 (#261), GPT-4 Turbo: 33.4 (#247)

BenchmarkClaude 3 OpusGPT-4 Turbo
WeirdML19.2%18%
BigCodeBench Instruct45.5%48.2%
LMArena Coding12641268
BigCodeBench Complete57.4%58.2%
HumanEval+77.4%86.6%
MBPP+73.3%73.3%
LiveBench Coding38.6%—

Agentic & Tool Use Not comparable

Claude 3 Opus: 24.3 (#110), GPT-4 Turbo: —

BenchmarkClaude 3 OpusGPT-4 Turbo
METR Time Horizons29.5%36.7%
Cybench10%—

Reasoning Too close to call

Claude 3 Opus: 14.2 (#318), GPT-4 Turbo: 14.9 (#311)

BenchmarkClaude 3 OpusGPT-4 Turbo
SimpleBench23.5%25.1%
Chess Puzzles5%6%
LMArena Hard Prompts12461251
DTBench61.6%61.6%
LMCA17%9.8%
Epoch Capabilities Index126.91127.25
ForecastBench58.459.4
EnigmaEval0.8%—
LiveBench Reasoning40.6%—
LiveBench Data Analysis57.9%—
LiveBench49.2%—
WinoGrande88.5%—

Math Claude 3 Opus leads

Claude 3 Opus: 13.9 (#293), GPT-4 Turbo: 8.6 (#316)

BenchmarkClaude 3 OpusGPT-4 Turbo
OTIS Mock AIME 2024-20254.7%6.7%
LMArena Math12731272
MATH Level 537.5%46.7%
FrontierMath (Tiers 1-3)—0.7%
LiveBench Math43.6%—

Knowledge Too close to call

Claude 3 Opus: 24.5 (#262), GPT-4 Turbo: 24.3 (#263)

BenchmarkClaude 3 OpusGPT-4 Turbo
GPQA Diamond47.2%46.6%
Confabulations22.7%28.4%
LMArena Expert12231223
MMLU84.6%81.3%
SimpleQA Verified12.6%—

Multimodal GPT-4 Turbo leads

Claude 3 Opus: 26.6 (#112), GPT-4 Turbo: 30.0 (#104)

BenchmarkClaude 3 OpusGPT-4 Turbo
LMArena Vision10231090

Multilingual Too close to call

Claude 3 Opus: 41.6 (#202), GPT-4 Turbo: 40.7 (#211)

BenchmarkClaude 3 OpusGPT-4 Turbo
LMArena Non-English12581245
LMArena Chinese12481242
LMArena French12741275
LMArena German12581259
LMArena Japanese12041194
LMArena Korean11871187
LMArena Russian12801259
LMArena Spanish12461260

Instruction Following GPT-4 Turbo leads

Claude 3 Opus: 64.3 (#223), GPT-4 Turbo: 66.0 (#211)

BenchmarkClaude 3 OpusGPT-4 Turbo
LMArena Instruction Following12481250
LiveBench Instruction Following63.9%—

Long Context Too close to call

Claude 3 Opus: 38.2 (#197), GPT-4 Turbo: 38.0 (#200)

BenchmarkClaude 3 OpusGPT-4 Turbo
LMArena Longer Query12591254

Writing & Preference Too close to call

Claude 3 Opus: 47.4 (#208), GPT-4 Turbo: 47.9 (#201)

BenchmarkClaude 3 OpusGPT-4 Turbo
LMArena Text12621272
LMArena Creative Writing12351269
LMArena Multi-Turn12751267
LiveBench Language50.4%—

Frequently asked questions

Is Claude 3 Opus better than GPT-4 Turbo?

GPT-4 Turbo is the stronger model overall, scoring 30.3 to 29.3 on the Noometry Index.

Is Claude 3 Opus or GPT-4 Turbo better for coding?

They score almost the same on coding (32.4 vs 33.4); test both on your own repository before choosing.

How many benchmarks do Claude 3 Opus and GPT-4 Turbo share?

35 benchmarks have published results for both models. Claude 3 Opus has 46 scored results on Noometry and GPT-4 Turbo has 36.

Related comparisons

Go deeper