Noometry

Model comparison

GPT-4 Turbo vs Llama 2-70B

GPT-4 Turbo is the stronger model overall, scoring 30.3 to 24.1 on the Noometry Index.

Last verified . 24 shared benchmarks.

GPT-4 Turbo OpenAI

30.3

Rank #281 Confirmed

Llama 2-70B Meta

24.1

Rank #343 Confirmed

Summary

  • They share 24 benchmarks with published results for both. GPT-4 Turbo scores higher in 8 categories and Llama 2-70B in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where GPT-4 Turbo leads 24.3 to 7.4.
  • The biggest single-benchmark swing is MATH Level 5: 46.7% for GPT-4 Turbo and 3.3% for Llama 2-70B.
  • Llama 2-70B has downloadable open weights; the other is API-only.

Side by side

GPT-4 TurboLlama 2-70B
ProviderOpenAIMeta
Noometry Index30.324.1
Released2023-11-062023-07-18
WeightsProprietaryOpen
Context window128K—
Max output4K—
Input $ / M tokens$10—
Output $ / M tokens$30—
Results tracked3635

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4 Turbo leads

GPT-4 Turbo: 33.4 (#247), Llama 2-70B: 30.8 (#281)

BenchmarkGPT-4 TurboLlama 2-70B
LMArena Coding12681079
WeirdML18%—
BigCodeBench Instruct48.2%—
BigCodeBench Complete58.2%—
HumanEval+86.6%—
MBPP+73.3%—

Agentic & Tool Use Not comparable

GPT-4 Turbo: —, Llama 2-70B: —

BenchmarkGPT-4 TurboLlama 2-70B
METR Time Horizons36.7%—

Reasoning GPT-4 Turbo leads

GPT-4 Turbo: 14.9 (#311), Llama 2-70B: 13.7 (#324)

BenchmarkGPT-4 TurboLlama 2-70B
LMArena Hard Prompts12511073
DTBench61.6%41.6%
Epoch Capabilities Index127.25113.59
ForecastBench59.451.4
SimpleBench25.1%—
Chess Puzzles6%—
LMCA9.8%—
BIG-Bench Hard—64.9%
CommonsenseQA 2.0—50%
HellaSwag—85.3%
LAMBADA—78.9%
PIQA—82.8%
WinoGrande—80.2%

Math GPT-4 Turbo leads

GPT-4 Turbo: 8.6 (#316), Llama 2-70B: 7.5 (#320)

BenchmarkGPT-4 TurboLlama 2-70B
OTIS Mock AIME 2024-20256.7%0%
LMArena Math12721091
MATH Level 546.7%3.3%
FrontierMath (Tiers 1-3)0.7%—
GSM8K—69.6%

Knowledge GPT-4 Turbo leads

GPT-4 Turbo: 24.3 (#263), Llama 2-70B: 7.4 (#305)

BenchmarkGPT-4 TurboLlama 2-70B
GPQA Diamond46.6%26.3%
LMArena Expert12231039
MMLU81.3%69.9%
Confabulations28.4%—
ARC (AI2) Challenge—78.3%
BoolQ—88.6%
OpenBookQA—60.2%
TriviaQA—87.6%

Multimodal Not comparable

GPT-4 Turbo: 30.0 (#104), Llama 2-70B: —

BenchmarkGPT-4 TurboLlama 2-70B
LMArena Vision1090—

Multilingual GPT-4 Turbo leads

GPT-4 Turbo: 40.7 (#211), Llama 2-70B: 27.9 (#269)

BenchmarkGPT-4 TurboLlama 2-70B
LMArena Non-English12451045
LMArena Chinese1242994
LMArena French12751089
LMArena German12591041
LMArena Japanese1194927
LMArena Korean1187964
LMArena Russian12591083
LMArena Spanish12601142

Instruction Following GPT-4 Turbo leads

GPT-4 Turbo: 66.0 (#211), Llama 2-70B: 55.1 (#273)

BenchmarkGPT-4 TurboLlama 2-70B
LMArena Instruction Following12501071

Long Context GPT-4 Turbo leads

GPT-4 Turbo: 38.0 (#200), Llama 2-70B: 32.2 (#265)

BenchmarkGPT-4 TurboLlama 2-70B
LMArena Longer Query12541062

Writing & Preference GPT-4 Turbo leads

GPT-4 Turbo: 47.9 (#201), Llama 2-70B: 32.5 (#274)

BenchmarkGPT-4 TurboLlama 2-70B
LMArena Text12721115
LMArena Creative Writing12691075
LMArena Multi-Turn12671088

Frequently asked questions

Is GPT-4 Turbo better than Llama 2-70B?

GPT-4 Turbo is the stronger model overall, scoring 30.3 to 24.1 on the Noometry Index.

Is GPT-4 Turbo or Llama 2-70B better for coding?

GPT-4 Turbo scores higher on coding benchmarks: 33.4 versus 30.8 in the Noometry coding category.

How many benchmarks do GPT-4 Turbo and Llama 2-70B share?

24 benchmarks have published results for both models. GPT-4 Turbo has 36 scored results on Noometry and Llama 2-70B has 35.

Related comparisons

Go deeper