Noometry

Model comparison

GPT-4 vs Llama 2-13B

GPT-4 and Llama 2-13B score almost the same on the Noometry Index (28.9 vs 29.1), so choose on price, context window or the category you care about most.

Last verified . 26 shared benchmarks.

GPT-4 OpenAI

28.9

Rank #308 Confirmed

Llama 2-13B Meta

29.1

Rank #304 Confirmed

Summary

  • They share 26 benchmarks with published results for both. GPT-4 scores higher in 6 categories and Llama 2-13B in 2 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Llama 2-13B leads 28.7 to 10.1.
  • The biggest single-benchmark swing is DTBench: 62.7% for GPT-4 and 42.2% for Llama 2-13B.
  • Llama 2-13B has downloadable open weights; the other is API-only.

Side by side

GPT-4Llama 2-13B
ProviderOpenAIMeta
Noometry Index28.929.1
Released2023-03-142023-07-18
WeightsProprietaryOpen
Context window8K—
Max output8K—
Input $ / M tokens$30—
Output $ / M tokens$60—
Results tracked3832

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

GPT-4: 31.2 (#277), Llama 2-13B: 30.4 (#287)

BenchmarkGPT-4Llama 2-13B
LMArena Coding12541062
WeirdML12.4%—
BigCodeBench Instruct46%—
BigCodeBench Complete57.2%—
HumanEval+79.3%—

Agentic & Tool Use Not comparable

GPT-4: —, Llama 2-13B: —

BenchmarkGPT-4Llama 2-13B
METR Time Horizons36.1%—

Reasoning GPT-4 leads

GPT-4: 17.3 (#285), Llama 2-13B: 12.3 (#333)

BenchmarkGPT-4Llama 2-13B
Chess Puzzles4%0%
LMArena Hard Prompts12411051
DTBench62.7%42.2%
BIG-Bench Hard75.1%58.2%
Epoch Capabilities Index125.88105.8
HellaSwag95.3%80.7%
WinoGrande87.5%72.8%
Mystery Game Puzzles12%—
LMCA17.1%—
ForecastBench57.8—
LAMBADA—76.5%
PIQA—80.8%

Math Llama 2-13B leads

GPT-4: 10.1 (#304), Llama 2-13B: 28.7 (#226)

BenchmarkGPT-4Llama 2-13B
LMArena Math12691065
GSM8K92%36.9%
OTIS Mock AIME 2024-20251.1%—
MATH Level 523%—

Knowledge Llama 2-13B leads

GPT-4: 18.4 (#277), Llama 2-13B: 27.5 (#244)

BenchmarkGPT-4Llama 2-13B
LMArena Expert12111030
MMLU86.4%55.6%
TriviaQA84.8%79.6%
GPQA Diamond35.7%—
ARC (AI2) Challenge—60.3%
BoolQ—82.4%
OpenBookQA—57%

Multimodal Not comparable

GPT-4: —, Llama 2-13B: —

BenchmarkGPT-4Llama 2-13B
ScienceQA—55.8%

Multilingual GPT-4 leads

GPT-4: 40.8 (#210), Llama 2-13B: 26.7 (#274)

BenchmarkGPT-4Llama 2-13B
LMArena Non-English12461024
LMArena Chinese12421001
LMArena French12821043
LMArena German12511009
LMArena Japanese1209894
LMArena Korean1184953
LMArena Russian12511055
LMArena Spanish12611086

Instruction Following GPT-4 leads

GPT-4: 65.5 (#217), Llama 2-13B: 53.5 (#282)

BenchmarkGPT-4Llama 2-13B
LMArena Instruction Following12411046

Long Context GPT-4 leads

GPT-4: 37.7 (#207), Llama 2-13B: 32.3 (#264)

BenchmarkGPT-4Llama 2-13B
LMArena Longer Query12441064

Writing & Preference GPT-4 leads

GPT-4: 35.0 (#263), Llama 2-13B: 30.0 (#284)

BenchmarkGPT-4Llama 2-13B
LMArena Text12631085
LMArena Creative Writing12441047
LMArena Multi-Turn12571050
EQ-Bench Creative Writing752—

Frequently asked questions

Is GPT-4 better than Llama 2-13B?

GPT-4 and Llama 2-13B score almost the same on the Noometry Index (28.9 vs 29.1), so choose on price, context window or the category you care about most.

Is GPT-4 or Llama 2-13B better for coding?

They score almost the same on coding (31.2 vs 30.4); test both on your own repository before choosing.

How many benchmarks do GPT-4 and Llama 2-13B share?

26 benchmarks have published results for both models. GPT-4 has 38 scored results on Noometry and Llama 2-13B has 32.

Related comparisons

Go deeper