Noometry

Model comparison

GPT-3.5-turbo vs Llama 2-7B

Llama 2-7B is the stronger model overall, scoring 29.1 to 23.2 on the Noometry Index.

Last verified . 25 shared benchmarks.

GPT-3.5-turbo OpenAI

23.2

Rank #350 Confirmed

Llama 2-7B Meta

29.1

Rank #317 Confirmed

Summary

  • They share 25 benchmarks with published results for both. GPT-3.5-turbo scores higher in 3 categories and Llama 2-7B in 5 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Llama 2-7B leads 30.7 to 6.3.
  • Llama 2-7B has downloadable open weights; the other is API-only.

Side by side

GPT-3.5-turboLlama 2-7B
ProviderOpenAIMeta
Noometry Index23.229.1
Released2023-03-012023-07-18
WeightsProprietaryOpen
Context window16K—
Max output4K—
Input $ / M tokens$0.50—
Output $ / M tokens$1.50—
Results tracked4429

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 2-7B leads

GPT-3.5-turbo: 23.9 (#331), Llama 2-7B: 29.2 (#307)

BenchmarkGPT-3.5-turboLlama 2-7B
LMArena Coding11361002
WeirdML3.5%—
BigCodeBench Instruct39.1%—
BigCodeBench Complete50.6%—
HumanEval+70.7%—
MBPP+69.7%—

Agentic & Tool Use Not comparable

GPT-3.5-turbo: —, Llama 2-7B: —

BenchmarkGPT-3.5-turboLlama 2-7B
METR Time Horizons21.5%—

Reasoning Llama 2-7B leads

GPT-3.5-turbo: 13.8 (#332), Llama 2-7B: 15.7 (#312)

BenchmarkGPT-3.5-turboLlama 2-7B
Chess Puzzles0%0%
LMArena Hard Prompts11081009
BIG-Bench Hard61.6%39.2%
Epoch Capabilities Index118.5599.06
WinoGrande81.6%69.2%
Mystery Game Puzzles3%—
DTBench48.5%—
LMCA9.7%—
Adversarial NLI58.1%—
CommonsenseQA 2.057%—
ForecastBench50.4—
HellaSwag—77.2%
LAMBADA—73.3%
PIQA—78.8%

Math Llama 2-7B leads

GPT-3.5-turbo: 6.3 (#327), Llama 2-7B: 30.7 (#233)

BenchmarkGPT-3.5-turboLlama 2-7B
LMArena Math11421042
GSM8K57.8%16.7%
FrontierMath (Tiers 1-3)0%—
OTIS Mock AIME 2024-20252.2%—
MATH Level 515.9%—

Knowledge Llama 2-7B leads

GPT-3.5-turbo: 10.0 (#303), Llama 2-7B: 28.2 (#248)

BenchmarkGPT-3.5-turboLlama 2-7B
LMArena Expert10701036
ARC (AI2) Challenge87.4%45.9%
BoolQ87%77.9%
MMLU71.4%45.8%
OpenBookQA86%58.6%
TriviaQA85.8%73.7%
GPQA Diamond28%—

Multimodal Not comparable

GPT-3.5-turbo: —, Llama 2-7B: —

BenchmarkGPT-3.5-turboLlama 2-7B
ScienceQA—43.1%

Multilingual GPT-3.5-turbo leads

GPT-3.5-turbo: 31.5 (#258), Llama 2-7B: 23.8 (#293)

BenchmarkGPT-3.5-turboLlama 2-7B
LMArena Non-English1108973
LMArena Chinese1075973
LMArena French1118970
LMArena German1090978
LMArena Russian1123995
LMArena Spanish11211007
LMArena Japanese1043—
LMArena Korean1019—

Instruction Following GPT-3.5-turbo leads

GPT-3.5-turbo: 57.9 (#262), Llama 2-7B: 50.8 (#298)

BenchmarkGPT-3.5-turboLlama 2-7B
LMArena Instruction Following11191006

Long Context GPT-3.5-turbo leads

GPT-3.5-turbo: 34.0 (#254), Llama 2-7B: 30.4 (#287)

BenchmarkGPT-3.5-turboLlama 2-7B
LMArena Longer Query1121999

Writing & Preference Llama 2-7B leads

GPT-3.5-turbo: 25.3 (#305), Llama 2-7B: 28.0 (#298)

BenchmarkGPT-3.5-turboLlama 2-7B
LMArena Text11251053
LMArena Creative Writing10921033
LMArena Multi-Turn11171029
EQ-Bench Creative Writing451—

Frequently asked questions

Is GPT-3.5-turbo better than Llama 2-7B?

Llama 2-7B is the stronger model overall, scoring 29.1 to 23.2 on the Noometry Index.

Is GPT-3.5-turbo or Llama 2-7B better for coding?

Llama 2-7B scores higher on coding benchmarks: 29.2 versus 23.9 in the Noometry coding category.

How many benchmarks do GPT-3.5-turbo and Llama 2-7B share?

25 benchmarks have published results for both models. GPT-3.5-turbo has 44 scored results on Noometry and Llama 2-7B has 29.

Related comparisons

Go deeper