Noometry

Model comparison

Claude 3 Sonnet vs GPT-4 Turbo

GPT-4 Turbo is the stronger model overall, scoring 30.5 to 29.0 on the Noometry Index.

Last verified . 29 shared benchmarks.

Claude 3 Sonnet Anthropic

29.0

Rank #319 Confirmed

GPT-4 Turbo OpenAI

30.5

Rank #292 Confirmed

Summary

  • They share 29 benchmarks with published results for both. Claude 3 Sonnet scores higher in 2 categories and GPT-4 Turbo in 7 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where GPT-4 Turbo leads 47.7 to 42.1.
  • The biggest single-benchmark swing is MATH Level 5: 18.2% for Claude 3 Sonnet and 46.7% for GPT-4 Turbo.

Side by side

Claude 3 SonnetGPT-4 Turbo
ProviderAnthropicOpenAI
Noometry Index29.030.5
Released2024-02-292023-11-06
WeightsProprietaryProprietary
Context window—128K
Max output—4K
Input $ / M tokens—$10
Output $ / M tokens—$30
Results tracked3036

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4 Turbo leads

Claude 3 Sonnet: 29.6 (#302), GPT-4 Turbo: 33.8 (#249)

BenchmarkClaude 3 SonnetGPT-4 Turbo
WeirdML10.2%18%
BigCodeBench Instruct42.7%48.2%
LMArena Coding12231268
BigCodeBench Complete53.8%58.2%
HumanEval+64%86.6%
MBPP+69.3%73.3%

Agentic & Tool Use Not comparable

Claude 3 Sonnet: —, GPT-4 Turbo: —

BenchmarkClaude 3 SonnetGPT-4 Turbo
METR Time Horizons—36.7%

Reasoning Claude 3 Sonnet leads

Claude 3 Sonnet: 20.5 (#237), GPT-4 Turbo: 15.3 (#317)

BenchmarkClaude 3 SonnetGPT-4 Turbo
LMArena Hard Prompts11971251
DTBench53.6%61.6%
Epoch Capabilities Index120.7127.25
SimpleBench—25.1%
Chess Puzzles—6%
LMCA—9.8%
ForecastBench—59.4
WinoGrande75.1%—

Math Claude 3 Sonnet leads

Claude 3 Sonnet: 10.7 (#310), GPT-4 Turbo: 9.0 (#322)

BenchmarkClaude 3 SonnetGPT-4 Turbo
OTIS Mock AIME 2024-20252.5%6.7%
LMArena Math12131272
MATH Level 518.2%46.7%
FrontierMath (Tiers 1-3)—0.7%

Knowledge GPT-4 Turbo leads

Claude 3 Sonnet: 21.1 (#276), GPT-4 Turbo: 24.3 (#268)

BenchmarkClaude 3 SonnetGPT-4 Turbo
GPQA Diamond40.6%46.6%
LMArena Expert11731223
MMLU75.9%81.3%
Confabulations—28.4%

Multimodal GPT-4 Turbo leads

Claude 3 Sonnet: 25.2 (#125), GPT-4 Turbo: 30.6 (#110)

BenchmarkClaude 3 SonnetGPT-4 Turbo
LMArena Vision9841090

Multilingual GPT-4 Turbo leads

Claude 3 Sonnet: 37.8 (#234), GPT-4 Turbo: 40.5 (#216)

BenchmarkClaude 3 SonnetGPT-4 Turbo
LMArena Non-English12051245
LMArena Chinese11891242
LMArena French12291276
LMArena German12041259
LMArena Japanese11311194
LMArena Korean11281187
LMArena Russian12271259
LMArena Spanish12041260

Instruction Following GPT-4 Turbo leads

Claude 3 Sonnet: 62.8 (#235), GPT-4 Turbo: 65.8 (#216)

BenchmarkClaude 3 SonnetGPT-4 Turbo
LMArena Instruction Following11991249

Long Context GPT-4 Turbo leads

Claude 3 Sonnet: 36.7 (#228), GPT-4 Turbo: 38.0 (#206)

BenchmarkClaude 3 SonnetGPT-4 Turbo
LMArena Longer Query12111254

Writing & Preference GPT-4 Turbo leads

Claude 3 Sonnet: 42.1 (#238), GPT-4 Turbo: 47.7 (#206)

BenchmarkClaude 3 SonnetGPT-4 Turbo
LMArena Text12181272
LMArena Creative Writing11861269
LMArena Multi-Turn12271267

Frequently asked questions

Is Claude 3 Sonnet better than GPT-4 Turbo?

GPT-4 Turbo is the stronger model overall, scoring 30.5 to 29.0 on the Noometry Index.

Is Claude 3 Sonnet or GPT-4 Turbo better for coding?

GPT-4 Turbo scores higher on coding benchmarks: 33.8 versus 29.6 in the Noometry coding category.

How many benchmarks do Claude 3 Sonnet and GPT-4 Turbo share?

29 benchmarks have published results for both models. Claude 3 Sonnet has 30 scored results on Noometry and GPT-4 Turbo has 36.

Related comparisons

Go deeper