Noometry

Model comparison

GPT-3.5-turbo vs Llama 3-70B

Llama 3-70B is the stronger model overall, scoring 28.8 to 23.2 on the Noometry Index.

Last verified . 29 shared benchmarks.

GPT-3.5-turbo OpenAI

23.2

Rank #350 Confirmed

Llama 3-70B Meta

28.8

Rank #323 Confirmed

Summary

  • They share 29 benchmarks with published results for both. GPT-3.5-turbo scores higher in 0 categories and Llama 3-70B in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Llama 3-70B leads 42.8 to 25.3.
  • The biggest single-benchmark swing is GPQA Diamond: 28% for GPT-3.5-turbo and 40.6% for Llama 3-70B.
  • Llama 3-70B has downloadable open weights; the other is API-only.

Side by side

GPT-3.5-turboLlama 3-70B
ProviderOpenAIMeta
Noometry Index23.228.8
Released2023-03-012024-04-18
WeightsProprietaryOpen
Context window16K—
Max output4K—
Input $ / M tokens$0.50—
Output $ / M tokens$1.50—
Results tracked4431

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3-70B leads

GPT-3.5-turbo: 23.9 (#331), Llama 3-70B: 35.8 (#218)

BenchmarkGPT-3.5-turboLlama 3-70B
BigCodeBench Instruct39.1%43.6%
LMArena Coding11361206
BigCodeBench Complete50.6%54.5%
HumanEval+70.7%72%
MBPP+69.7%69%
WeirdML3.5%—

Agentic & Tool Use Not comparable

GPT-3.5-turbo: —, Llama 3-70B: 21.1 (#139)

BenchmarkGPT-3.5-turboLlama 3-70B
Cybench—5%
METR Time Horizons21.5%—

Reasoning Llama 3-70B leads

GPT-3.5-turbo: 13.8 (#332), Llama 3-70B: 18.0 (#288)

BenchmarkGPT-3.5-turboLlama 3-70B
LMArena Hard Prompts11081195
DTBench48.5%54.2%
Epoch Capabilities Index118.55122.93
ForecastBench50.457.1
WinoGrande81.6%83.5%
Kagi LLM Benchmark—35.1%
Chess Puzzles0%—
Mystery Game Puzzles3%—
LMCA9.7%—
Adversarial NLI58.1%—
BIG-Bench Hard61.6%—
CommonsenseQA 2.057%—

Math Llama 3-70B leads

GPT-3.5-turbo: 6.3 (#327), Llama 3-70B: 12.8 (#305)

BenchmarkGPT-3.5-turboLlama 3-70B
OTIS Mock AIME 2024-20252.2%4.3%
LMArena Math11421218
MATH Level 515.9%22.6%
FrontierMath (Tiers 1-3)0%—
GSM8K57.8%—

Knowledge Llama 3-70B leads

GPT-3.5-turbo: 10.0 (#303), Llama 3-70B: 20.8 (#277)

BenchmarkGPT-3.5-turboLlama 3-70B
GPQA Diamond28%40.6%
LMArena Expert10701149
MMLU71.4%79.3%
ARC (AI2) Challenge87.4%—
BoolQ87%—
OpenBookQA86%—
TriviaQA85.8%—

Multilingual Llama 3-70B leads

GPT-3.5-turbo: 31.5 (#258), Llama 3-70B: 33.6 (#251)

BenchmarkGPT-3.5-turboLlama 3-70B
LMArena Non-English11081142
LMArena Chinese10751114
LMArena French11181232
LMArena German10901169
LMArena Japanese10431017
LMArena Korean10191017
LMArena Russian11231159
LMArena Spanish11211241

Instruction Following Llama 3-70B leads

GPT-3.5-turbo: 57.9 (#262), Llama 3-70B: 62.5 (#238)

BenchmarkGPT-3.5-turboLlama 3-70B
LMArena Instruction Following11191194

Long Context Llama 3-70B leads

GPT-3.5-turbo: 34.0 (#254), Llama 3-70B: 35.6 (#240)

BenchmarkGPT-3.5-turboLlama 3-70B
LMArena Longer Query11211174

Writing & Preference Llama 3-70B leads

GPT-3.5-turbo: 25.3 (#305), Llama 3-70B: 42.8 (#231)

BenchmarkGPT-3.5-turboLlama 3-70B
LMArena Text11251221
LMArena Creative Writing10921210
LMArena Multi-Turn11171223
EQ-Bench Creative Writing451—

Frequently asked questions

Is GPT-3.5-turbo better than Llama 3-70B?

Llama 3-70B is the stronger model overall, scoring 28.8 to 23.2 on the Noometry Index.

Is GPT-3.5-turbo or Llama 3-70B better for coding?

Llama 3-70B scores higher on coding benchmarks: 35.8 versus 23.9 in the Noometry coding category.

How many benchmarks do GPT-3.5-turbo and Llama 3-70B share?

29 benchmarks have published results for both models. GPT-3.5-turbo has 44 scored results on Noometry and Llama 3-70B has 31.

Related comparisons

Go deeper