Noometry

Model comparison

GPT-4 Turbo vs Llama 3-70B

GPT-4 Turbo is the stronger model overall, scoring 30.5 to 28.8 on the Noometry Index.

Last verified . 28 shared benchmarks.

GPT-4 Turbo OpenAI

30.5

Rank #292 Confirmed

Llama 3-70B Meta

28.8

Rank #323 Confirmed

Summary

  • They share 28 benchmarks with published results for both. GPT-4 Turbo scores higher in 5 categories and Llama 3-70B in 3 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in multilingual, where GPT-4 Turbo leads 40.5 to 33.6.
  • The biggest single-benchmark swing is MATH Level 5: 46.7% for GPT-4 Turbo and 22.6% for Llama 3-70B.
  • Llama 3-70B has downloadable open weights; the other is API-only.

Side by side

GPT-4 TurboLlama 3-70B
ProviderOpenAIMeta
Noometry Index30.528.8
Released2023-11-062024-04-18
WeightsProprietaryOpen
Context window128K—
Max output4K—
Input $ / M tokens$10—
Output $ / M tokens$30—
Results tracked3631

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3-70B leads

GPT-4 Turbo: 33.8 (#249), Llama 3-70B: 35.8 (#218)

BenchmarkGPT-4 TurboLlama 3-70B
BigCodeBench Instruct48.2%43.6%
LMArena Coding12681206
BigCodeBench Complete58.2%54.5%
HumanEval+86.6%72%
MBPP+73.3%69%
WeirdML18%—

Agentic & Tool Use Not comparable

GPT-4 Turbo: —, Llama 3-70B: 21.1 (#139)

BenchmarkGPT-4 TurboLlama 3-70B
Cybench—5%
METR Time Horizons36.7%—

Reasoning Llama 3-70B leads

GPT-4 Turbo: 15.3 (#317), Llama 3-70B: 18.0 (#288)

BenchmarkGPT-4 TurboLlama 3-70B
LMArena Hard Prompts12511195
DTBench61.6%54.2%
Epoch Capabilities Index127.25122.93
ForecastBench59.457.1
SimpleBench25.1%—
Kagi LLM Benchmark—35.1%
Chess Puzzles6%—
LMCA9.8%—
WinoGrande—83.5%

Math Llama 3-70B leads

GPT-4 Turbo: 9.0 (#322), Llama 3-70B: 12.8 (#305)

BenchmarkGPT-4 TurboLlama 3-70B
OTIS Mock AIME 2024-20256.7%4.3%
LMArena Math12721218
MATH Level 546.7%22.6%
FrontierMath (Tiers 1-3)0.7%—

Knowledge GPT-4 Turbo leads

GPT-4 Turbo: 24.3 (#268), Llama 3-70B: 20.8 (#277)

BenchmarkGPT-4 TurboLlama 3-70B
GPQA Diamond46.6%40.6%
LMArena Expert12231149
MMLU81.3%79.3%
Confabulations28.4%—

Multimodal Not comparable

GPT-4 Turbo: 30.6 (#110), Llama 3-70B: —

BenchmarkGPT-4 TurboLlama 3-70B
LMArena Vision1090—

Multilingual GPT-4 Turbo leads

GPT-4 Turbo: 40.5 (#216), Llama 3-70B: 33.6 (#251)

BenchmarkGPT-4 TurboLlama 3-70B
LMArena Non-English12451142
LMArena Chinese12421114
LMArena French12761232
LMArena German12591169
LMArena Japanese11941017
LMArena Korean11871017
LMArena Russian12591159
LMArena Spanish12601241

Instruction Following GPT-4 Turbo leads

GPT-4 Turbo: 65.8 (#216), Llama 3-70B: 62.5 (#238)

BenchmarkGPT-4 TurboLlama 3-70B
LMArena Instruction Following12491194

Long Context GPT-4 Turbo leads

GPT-4 Turbo: 38.0 (#206), Llama 3-70B: 35.6 (#240)

BenchmarkGPT-4 TurboLlama 3-70B
LMArena Longer Query12541174

Writing & Preference GPT-4 Turbo leads

GPT-4 Turbo: 47.7 (#206), Llama 3-70B: 42.8 (#231)

BenchmarkGPT-4 TurboLlama 3-70B
LMArena Text12721221
LMArena Creative Writing12691210
LMArena Multi-Turn12671223

Frequently asked questions

Is GPT-4 Turbo better than Llama 3-70B?

GPT-4 Turbo is the stronger model overall, scoring 30.5 to 28.8 on the Noometry Index.

Is GPT-4 Turbo or Llama 3-70B better for coding?

Llama 3-70B scores higher on coding benchmarks: 35.8 versus 33.8 in the Noometry coding category.

How many benchmarks do GPT-4 Turbo and Llama 3-70B share?

28 benchmarks have published results for both models. GPT-4 Turbo has 36 scored results on Noometry and Llama 3-70B has 31.

Related comparisons

Go deeper