Noometry

Model comparison

Llama 2-13B vs Llama 3.1-405B

Llama 3.1-405B is the stronger model overall, scoring 30.7 to 29.6 on the Noometry Index.

Last verified . 26 shared benchmarks.

Llama 2-13B Meta

29.6

Rank #309 Confirmed

Llama 3.1-405B Meta

30.7

Rank #288 Confirmed

Summary

  • They share 26 benchmarks with published results for both. Llama 2-13B scores higher in 1 category and Llama 3.1-405B in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in multilingual, where Llama 3.1-405B leads 40.7 to 26.5.
  • The biggest single-benchmark swing is DTBench: 42.2% for Llama 2-13B and 61.4% for Llama 3.1-405B.

Side by side

Llama 2-13BLlama 3.1-405B
ProviderMetaMeta
Noometry Index29.630.7
Released2023-07-182024-07-23
WeightsOpenOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3242

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3.1-405B leads

Llama 2-13B: 30.9 (#291), Llama 3.1-405B: 33.1 (#262)

BenchmarkLlama 2-13BLlama 3.1-405B
LMArena Coding10621291
WeirdML—21.4%

Agentic & Tool Use Not comparable

Llama 2-13B: —, Llama 3.1-405B: 21.0 (#140)

BenchmarkLlama 2-13BLlama 3.1-405B
TheAgentCompany—7.4%
Cybench—7.5%

Reasoning Llama 3.1-405B leads

Llama 2-13B: 12.8 (#337), Llama 3.1-405B: 16.8 (#300)

BenchmarkLlama 2-13BLlama 3.1-405B
LMArena Hard Prompts10511269
DTBench42.2%61.4%
BIG-Bench Hard58.2%82.9%
Epoch Capabilities Index106.17128.75
HellaSwag80.7%89.2%
PIQA80.8%85.9%
WinoGrande72.8%89.2%
SimpleBench—23%
Kagi LLM Benchmark—45%
Chess Puzzles0%—
ForecastBench—59.9
LAMBADA76.5%—

Math Llama 2-13B leads

Llama 2-13B: 31.1 (#229), Llama 3.1-405B: 18.4 (#290)

BenchmarkLlama 2-13BLlama 3.1-405B
LMArena Math10651281
OTIS Mock AIME 2024-2025—9.7%
Omni-MATH—24.9%
MATH Level 5—49.8%
GSM8K36.9%—

Knowledge Llama 3.1-405B leads

Llama 2-13B: 28.1 (#249), Llama 3.1-405B: 30.4 (#227)

BenchmarkLlama 2-13BLlama 3.1-405B
LMArena Expert10301243
ARC (AI2) Challenge60.3%95.3%
MMLU55.6%84.5%
TriviaQA79.6%82.7%
GPQA Diamond—50.9%
MMLU-Pro—72.3%
Confabulations—17.6%
GPQA (HELM)—52.2%
BoolQ82.4%—
OpenBookQA57%—

Multimodal Not comparable

Llama 2-13B: —, Llama 3.1-405B: —

BenchmarkLlama 2-13BLlama 3.1-405B
ScienceQA55.8%—

Multilingual Llama 3.1-405B leads

Llama 2-13B: 26.5 (#279), Llama 3.1-405B: 40.7 (#214)

BenchmarkLlama 2-13BLlama 3.1-405B
LMArena Non-English10241248
LMArena Chinese10011242
LMArena French10441279
LMArena German10091252
LMArena Japanese8941208
LMArena Korean9531184
LMArena Russian10551265
LMArena Spanish10871260

Instruction Following Llama 3.1-405B leads

Llama 2-13B: 53.3 (#287), Llama 3.1-405B: 65.9 (#214)

BenchmarkLlama 2-13BLlama 3.1-405B
LMArena Instruction Following10451259
IFEval—81.1%

Long Context Llama 3.1-405B leads

Llama 2-13B: 32.3 (#269), Llama 3.1-405B: 38.4 (#197)

BenchmarkLlama 2-13BLlama 3.1-405B
LMArena Longer Query10641266

Writing & Preference Llama 3.1-405B leads

Llama 2-13B: 29.8 (#289), Llama 3.1-405B: 38.9 (#251)

BenchmarkLlama 2-13BLlama 3.1-405B
LMArena Text10841284
LMArena Creative Writing10471262
LMArena Multi-Turn10501297
EQ-Bench Creative Writing—870
WildBench—78.3%

Frequently asked questions

Is Llama 2-13B better than Llama 3.1-405B?

Llama 3.1-405B is the stronger model overall, scoring 30.7 to 29.6 on the Noometry Index.

Is Llama 2-13B or Llama 3.1-405B better for coding?

Llama 3.1-405B scores higher on coding benchmarks: 33.1 versus 30.9 in the Noometry coding category.

How many benchmarks do Llama 2-13B and Llama 3.1-405B share?

26 benchmarks have published results for both models. Llama 2-13B has 32 scored results on Noometry and Llama 3.1-405B has 42.

Related comparisons

Go deeper