Noometry

Model comparison

Amazon Nova Pro vs Llama 13b

Amazon Nova Pro is the stronger model overall, scoring 31.0 to 24.4 on the Noometry Index.

Last verified . 10 shared benchmarks.

Amazon Nova Pro Amazon

31.0

Rank #281 Confirmed

Llama 13b Meta

24.4

Rank #348 Confirmed

Summary

  • They share 10 benchmarks with published results for both. Amazon Nova Pro scores higher in 6 categories and Llama 13b in 0 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Amazon Nova Pro leads 43.9 to 13.8.
  • Llama 13b has downloadable open weights; the other is API-only.

Side by side

Amazon Nova ProLlama 13b
ProviderAmazonMeta
Noometry Index31.024.4
Released2024-12-032023-02-24
WeightsProprietaryOpen
Context window300K—
Max output10K—
Input $ / M tokens$0.80—
Output $ / M tokens$3.20—
Results tracked3821

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Amazon Nova Pro leads

Amazon Nova Pro: 35.1 (#229), Llama 13b: 21.4 (#337)

BenchmarkAmazon Nova ProLlama 13b
LMArena Coding1270683
LiveBench Coding38.1%—

Agentic & Tool Use Not comparable

Amazon Nova Pro: 16.7 (#147), Llama 13b: —

BenchmarkAmazon Nova ProLlama 13b
Berkeley Function Calling Leaderboard25%—
TheAgentCompany1.7%—

Reasoning Amazon Nova Pro leads

Amazon Nova Pro: 20.0 (#243), Llama 13b: 14.0 (#329)

BenchmarkAmazon Nova ProLlama 13b
LMArena Hard Prompts1246728
Epoch Capabilities Index123.8100.58
LiveBench Reasoning32.6%—
LiveBench Data Analysis48.3%—
BIG-Bench Hard—37.9%
HellaSwag—79.2%
LAMBADA—75.2%
LiveBench43.5%—
PIQA—80.1%
WinoGrande—73%

Math Amazon Nova Pro leads

Amazon Nova Pro: 28.5 (#243), Llama 13b: 26.7 (#256)

BenchmarkAmazon Nova ProLlama 13b
LMArena Math1252838
Omni-MATH24.2%—
LiveBench Math38%—
GSM8K—20.6%

Knowledge Not comparable

Amazon Nova Pro: 27.4 (#250), Llama 13b: —

BenchmarkAmazon Nova ProLlama 13b
MMLU82%47.7%
Humanity's Last Exam4.4%—
MMLU-Pro67.3%—
Confabulations30.1%—
Vectara Hallucination Rate5.1%—
GPQA (HELM)44.6%—
LMArena Expert1211—
ARC (AI2) Challenge—52.7%
BoolQ—78.7%
OpenBookQA—56.4%
TriviaQA—77.9%

Multimodal Not comparable

Amazon Nova Pro: 25.0 (#126), Llama 13b: —

BenchmarkAmazon Nova ProLlama 13b
LMArena Vision980—
ScienceQA—43.3%

Multilingual Amazon Nova Pro leads

Amazon Nova Pro: 39.7 (#223), Llama 13b: 16.6 (#297)

BenchmarkAmazon Nova ProLlama 13b
LMArena Non-English1234819
LMArena Chinese1244—
LMArena French1271—
LMArena German1243—
LMArena Japanese1200—
LMArena Korean1203—
LMArena Russian1240—
LMArena Spanish1182—

Instruction Following Amazon Nova Pro leads

Amazon Nova Pro: 64.9 (#226), Llama 13b: 36.7 (#305)

BenchmarkAmazon Nova ProLlama 13b
LMArena Instruction Following1235781
LiveBench Instruction Following67.1%—
IFEval81.5%—

Long Context Not comparable

Amazon Nova Pro: 38.1 (#205), Llama 13b: —

BenchmarkAmazon Nova ProLlama 13b
LMArena Longer Query1255—

Writing & Preference Amazon Nova Pro leads

Amazon Nova Pro: 43.9 (#226), Llama 13b: 13.8 (#312)

BenchmarkAmazon Nova ProLlama 13b
LMArena Text1259834
LMArena Creative Writing1212794
LMArena Multi-Turn1246753
Short-Story Creative Writing60.5%—
WildBench77.7%—
LiveBench Language37%—

Frequently asked questions

Is Amazon Nova Pro better than Llama 13b?

Amazon Nova Pro is the stronger model overall, scoring 31.0 to 24.4 on the Noometry Index.

Is Amazon Nova Pro or Llama 13b better for coding?

Amazon Nova Pro scores higher on coding benchmarks: 35.1 versus 21.4 in the Noometry coding category.

How many benchmarks do Amazon Nova Pro and Llama 13b share?

10 benchmarks have published results for both models. Amazon Nova Pro has 38 scored results on Noometry and Llama 13b has 21.

Related comparisons

Go deeper