Noometry

Model comparison

GPT-4 Turbo vs Llama 2-7B

GPT-4 Turbo is the stronger model overall, scoring 30.5 to 29.1 on the Noometry Index.

Last verified . 18 shared benchmarks.

GPT-4 Turbo OpenAI

30.5

Rank #292 Confirmed

Llama 2-7B Meta

29.1

Rank #317 Confirmed

Summary

  • They share 18 benchmarks with published results for both. GPT-4 Turbo scores higher in 5 categories and Llama 2-7B in 3 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Llama 2-7B leads 30.7 to 9.0.
  • The biggest single-benchmark swing is Chess Puzzles: 6% for GPT-4 Turbo and 0% for Llama 2-7B.
  • Llama 2-7B has downloadable open weights; the other is API-only.

Side by side

GPT-4 TurboLlama 2-7B
ProviderOpenAIMeta
Noometry Index30.529.1
Released2023-11-062023-07-18
WeightsProprietaryOpen
Context window128K—
Max output4K—
Input $ / M tokens$10—
Output $ / M tokens$30—
Results tracked3629

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4 Turbo leads

GPT-4 Turbo: 33.8 (#249), Llama 2-7B: 29.2 (#307)

BenchmarkGPT-4 TurboLlama 2-7B
LMArena Coding12681002
WeirdML18%—
BigCodeBench Instruct48.2%—
BigCodeBench Complete58.2%—
HumanEval+86.6%—
MBPP+73.3%—

Agentic & Tool Use Not comparable

GPT-4 Turbo: —, Llama 2-7B: —

BenchmarkGPT-4 TurboLlama 2-7B
METR Time Horizons36.7%—

Reasoning Too close to call

GPT-4 Turbo: 15.3 (#317), Llama 2-7B: 15.7 (#312)

BenchmarkGPT-4 TurboLlama 2-7B
Chess Puzzles6%0%
LMArena Hard Prompts12511009
Epoch Capabilities Index127.2599.06
SimpleBench25.1%—
DTBench61.6%—
LMCA9.8%—
BIG-Bench Hard—39.2%
ForecastBench59.4—
HellaSwag—77.2%
LAMBADA—73.3%
PIQA—78.8%
WinoGrande—69.2%

Math Llama 2-7B leads

GPT-4 Turbo: 9.0 (#322), Llama 2-7B: 30.7 (#233)

BenchmarkGPT-4 TurboLlama 2-7B
LMArena Math12721042
FrontierMath (Tiers 1-3)0.7%—
OTIS Mock AIME 2024-20256.7%—
MATH Level 546.7%—
GSM8K—16.7%

Knowledge Llama 2-7B leads

GPT-4 Turbo: 24.3 (#268), Llama 2-7B: 28.2 (#248)

BenchmarkGPT-4 TurboLlama 2-7B
LMArena Expert12231036
MMLU81.3%45.8%
GPQA Diamond46.6%—
Confabulations28.4%—
ARC (AI2) Challenge—45.9%
BoolQ—77.9%
OpenBookQA—58.6%
TriviaQA—73.7%

Multimodal Not comparable

GPT-4 Turbo: 30.6 (#110), Llama 2-7B: —

BenchmarkGPT-4 TurboLlama 2-7B
LMArena Vision1090—
ScienceQA—43.1%

Multilingual GPT-4 Turbo leads

GPT-4 Turbo: 40.5 (#216), Llama 2-7B: 23.8 (#293)

BenchmarkGPT-4 TurboLlama 2-7B
LMArena Non-English1245973
LMArena Chinese1242973
LMArena French1276970
LMArena German1259978
LMArena Russian1259995
LMArena Spanish12601007
LMArena Japanese1194—
LMArena Korean1187—

Instruction Following GPT-4 Turbo leads

GPT-4 Turbo: 65.8 (#216), Llama 2-7B: 50.8 (#298)

BenchmarkGPT-4 TurboLlama 2-7B
LMArena Instruction Following12491006

Long Context GPT-4 Turbo leads

GPT-4 Turbo: 38.0 (#206), Llama 2-7B: 30.4 (#287)

BenchmarkGPT-4 TurboLlama 2-7B
LMArena Longer Query1254999

Writing & Preference GPT-4 Turbo leads

GPT-4 Turbo: 47.7 (#206), Llama 2-7B: 28.0 (#298)

BenchmarkGPT-4 TurboLlama 2-7B
LMArena Text12721053
LMArena Creative Writing12691033
LMArena Multi-Turn12671029

Frequently asked questions

Is GPT-4 Turbo better than Llama 2-7B?

GPT-4 Turbo is the stronger model overall, scoring 30.5 to 29.1 on the Noometry Index.

Is GPT-4 Turbo or Llama 2-7B better for coding?

GPT-4 Turbo scores higher on coding benchmarks: 33.8 versus 29.2 in the Noometry coding category.

How many benchmarks do GPT-4 Turbo and Llama 2-7B share?

18 benchmarks have published results for both models. GPT-4 Turbo has 36 scored results on Noometry and Llama 2-7B has 29.

Related comparisons

Go deeper