Noometry

Model comparison

Claude 3 Sonnet vs GPT-3.5-turbo

Claude 3 Sonnet is the stronger model overall, scoring 29.0 to 23.2 on the Noometry Index.

Last verified . 29 shared benchmarks.

Claude 3 Sonnet Anthropic

29.0

Rank #319 Confirmed

GPT-3.5-turbo OpenAI

23.2

Rank #350 Confirmed

Summary

  • They share 29 benchmarks with published results for both. Claude 3 Sonnet scores higher in 8 categories and GPT-3.5-turbo in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Claude 3 Sonnet leads 42.1 to 25.3.
  • The biggest single-benchmark swing is GPQA Diamond: 40.6% for Claude 3 Sonnet and 28% for GPT-3.5-turbo.

Side by side

Claude 3 SonnetGPT-3.5-turbo
ProviderAnthropicOpenAI
Noometry Index29.023.2
Released2024-02-292023-03-01
WeightsProprietaryProprietary
Context window—16K
Max output—4K
Input $ / M tokens—$0.50
Output $ / M tokens—$1.50
Results tracked3044

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude 3 Sonnet leads

Claude 3 Sonnet: 29.6 (#302), GPT-3.5-turbo: 23.9 (#331)

BenchmarkClaude 3 SonnetGPT-3.5-turbo
WeirdML10.2%3.5%
BigCodeBench Instruct42.7%39.1%
LMArena Coding12231136
BigCodeBench Complete53.8%50.6%
HumanEval+64%70.7%
MBPP+69.3%69.7%

Agentic & Tool Use Not comparable

Claude 3 Sonnet: —, GPT-3.5-turbo: —

BenchmarkClaude 3 SonnetGPT-3.5-turbo
METR Time Horizons—21.5%

Reasoning Claude 3 Sonnet leads

Claude 3 Sonnet: 20.5 (#237), GPT-3.5-turbo: 13.8 (#332)

BenchmarkClaude 3 SonnetGPT-3.5-turbo
LMArena Hard Prompts11971108
DTBench53.6%48.5%
Epoch Capabilities Index120.7118.55
WinoGrande75.1%81.6%
Chess Puzzles—0%
Mystery Game Puzzles—3%
LMCA—9.7%
Adversarial NLI—58.1%
BIG-Bench Hard—61.6%
CommonsenseQA 2.0—57%
ForecastBench—50.4

Math Claude 3 Sonnet leads

Claude 3 Sonnet: 10.7 (#310), GPT-3.5-turbo: 6.3 (#327)

BenchmarkClaude 3 SonnetGPT-3.5-turbo
OTIS Mock AIME 2024-20252.5%2.2%
LMArena Math12131142
MATH Level 518.2%15.9%
FrontierMath (Tiers 1-3)—0%
GSM8K—57.8%

Knowledge Claude 3 Sonnet leads

Claude 3 Sonnet: 21.1 (#276), GPT-3.5-turbo: 10.0 (#303)

BenchmarkClaude 3 SonnetGPT-3.5-turbo
GPQA Diamond40.6%28%
LMArena Expert11731070
MMLU75.9%71.4%
ARC (AI2) Challenge—87.4%
BoolQ—87%
OpenBookQA—86%
TriviaQA—85.8%

Multimodal Not comparable

Claude 3 Sonnet: 25.2 (#125), GPT-3.5-turbo: —

BenchmarkClaude 3 SonnetGPT-3.5-turbo
LMArena Vision984—

Multilingual Claude 3 Sonnet leads

Claude 3 Sonnet: 37.8 (#234), GPT-3.5-turbo: 31.5 (#258)

BenchmarkClaude 3 SonnetGPT-3.5-turbo
LMArena Non-English12051108
LMArena Chinese11891075
LMArena French12291118
LMArena German12041090
LMArena Japanese11311043
LMArena Korean11281019
LMArena Russian12271123
LMArena Spanish12041121

Instruction Following Claude 3 Sonnet leads

Claude 3 Sonnet: 62.8 (#235), GPT-3.5-turbo: 57.9 (#262)

BenchmarkClaude 3 SonnetGPT-3.5-turbo
LMArena Instruction Following11991119

Long Context Claude 3 Sonnet leads

Claude 3 Sonnet: 36.7 (#228), GPT-3.5-turbo: 34.0 (#254)

BenchmarkClaude 3 SonnetGPT-3.5-turbo
LMArena Longer Query12111121

Writing & Preference Claude 3 Sonnet leads

Claude 3 Sonnet: 42.1 (#238), GPT-3.5-turbo: 25.3 (#305)

BenchmarkClaude 3 SonnetGPT-3.5-turbo
LMArena Text12181125
LMArena Creative Writing11861092
LMArena Multi-Turn12271117
EQ-Bench Creative Writing—451

Frequently asked questions

Is Claude 3 Sonnet better than GPT-3.5-turbo?

Claude 3 Sonnet is the stronger model overall, scoring 29.0 to 23.2 on the Noometry Index.

Is Claude 3 Sonnet or GPT-3.5-turbo better for coding?

Claude 3 Sonnet scores higher on coding benchmarks: 29.6 versus 23.9 in the Noometry coding category.

How many benchmarks do Claude 3 Sonnet and GPT-3.5-turbo share?

29 benchmarks have published results for both models. Claude 3 Sonnet has 30 scored results on Noometry and GPT-3.5-turbo has 44.

Related comparisons

Go deeper