Noometry

Model comparison

GPT-3.5-turbo vs Llama 2-70B

Llama 2-70B is the stronger model overall, scoring 24.4 to 23.2 on the Noometry Index.

Last verified . 32 shared benchmarks.

GPT-3.5-turbo OpenAI

23.2

Rank #350 Confirmed

Llama 2-70B Meta

24.4

Rank #349 Confirmed

Summary

  • They share 32 benchmarks with published results for both. GPT-3.5-turbo scores higher in 4 categories and Llama 2-70B in 4 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in coding, where Llama 2-70B leads 31.4 to 23.9.
  • The biggest single-benchmark swing is MATH Level 5: 15.9% for GPT-3.5-turbo and 3.3% for Llama 2-70B.
  • Llama 2-70B has downloadable open weights; the other is API-only.

Side by side

GPT-3.5-turboLlama 2-70B
ProviderOpenAIMeta
Noometry Index23.224.4
Released2023-03-012023-07-18
WeightsProprietaryOpen
Context window16K—
Max output4K—
Input $ / M tokens$0.50—
Output $ / M tokens$1.50—
Results tracked4435

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 2-70B leads

GPT-3.5-turbo: 23.9 (#331), Llama 2-70B: 31.4 (#286)

BenchmarkGPT-3.5-turboLlama 2-70B
LMArena Coding11361079
WeirdML3.5%—
BigCodeBench Instruct39.1%—
BigCodeBench Complete50.6%—
HumanEval+70.7%—
MBPP+69.7%—

Agentic & Tool Use Not comparable

GPT-3.5-turbo: —, Llama 2-70B: —

BenchmarkGPT-3.5-turboLlama 2-70B
METR Time Horizons21.5%—

Reasoning Too close to call

GPT-3.5-turbo: 13.8 (#332), Llama 2-70B: 14.4 (#325)

BenchmarkGPT-3.5-turboLlama 2-70B
LMArena Hard Prompts11081073
DTBench48.5%41.6%
BIG-Bench Hard61.6%64.9%
CommonsenseQA 2.057%50%
Epoch Capabilities Index118.55113.79
ForecastBench50.451.4
WinoGrande81.6%80.2%
Chess Puzzles0%—
Mystery Game Puzzles3%—
LMCA9.7%—
Adversarial NLI58.1%—
HellaSwag—85.3%
LAMBADA—78.9%
PIQA—82.8%

Math Llama 2-70B leads

GPT-3.5-turbo: 6.3 (#327), Llama 2-70B: 8.1 (#326)

BenchmarkGPT-3.5-turboLlama 2-70B
OTIS Mock AIME 2024-20252.2%0%
LMArena Math11421091
MATH Level 515.9%3.3%
GSM8K57.8%69.6%
FrontierMath (Tiers 1-3)0%—

Knowledge GPT-3.5-turbo leads

GPT-3.5-turbo: 10.0 (#303), Llama 2-70B: 7.4 (#310)

BenchmarkGPT-3.5-turboLlama 2-70B
GPQA Diamond28%26.3%
LMArena Expert10701039
ARC (AI2) Challenge87.4%78.3%
BoolQ87%88.6%
MMLU71.4%69.9%
OpenBookQA86%60.2%
TriviaQA85.8%87.6%

Multilingual GPT-3.5-turbo leads

GPT-3.5-turbo: 31.5 (#258), Llama 2-70B: 27.7 (#274)

BenchmarkGPT-3.5-turboLlama 2-70B
LMArena Non-English11081045
LMArena Chinese1075995
LMArena French11181090
LMArena German10901041
LMArena Japanese1043927
LMArena Korean1019964
LMArena Russian11231083
LMArena Spanish11211143

Instruction Following GPT-3.5-turbo leads

GPT-3.5-turbo: 57.9 (#262), Llama 2-70B: 54.9 (#278)

BenchmarkGPT-3.5-turboLlama 2-70B
LMArena Instruction Following11191071

Long Context GPT-3.5-turbo leads

GPT-3.5-turbo: 34.0 (#254), Llama 2-70B: 32.3 (#270)

BenchmarkGPT-3.5-turboLlama 2-70B
LMArena Longer Query11211062

Writing & Preference Llama 2-70B leads

GPT-3.5-turbo: 25.3 (#305), Llama 2-70B: 32.3 (#279)

BenchmarkGPT-3.5-turboLlama 2-70B
LMArena Text11251115
LMArena Creative Writing10921075
LMArena Multi-Turn11171088
EQ-Bench Creative Writing451—

Frequently asked questions

Is GPT-3.5-turbo better than Llama 2-70B?

Llama 2-70B is the stronger model overall, scoring 24.4 to 23.2 on the Noometry Index.

Is GPT-3.5-turbo or Llama 2-70B better for coding?

Llama 2-70B scores higher on coding benchmarks: 31.4 versus 23.9 in the Noometry coding category.

How many benchmarks do GPT-3.5-turbo and Llama 2-70B share?

32 benchmarks have published results for both models. GPT-3.5-turbo has 44 scored results on Noometry and Llama 2-70B has 35.

Related comparisons

Go deeper