Noometry

Model comparison

Amazon Nova Pro vs Llama 3-8B

Amazon Nova Pro is the stronger model overall, scoring 31.0 to 25.5 on the Noometry Index.

Last verified . 19 shared benchmarks.

Amazon Nova Pro Amazon

31.0

Rank #281 Confirmed

Llama 3-8B Meta

25.5

Rank #344 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Amazon Nova Pro scores higher in 8 categories and Llama 3-8B in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Amazon Nova Pro leads 28.5 to 8.8.
  • Llama 3-8B has downloadable open weights; the other is API-only.

Side by side

Amazon Nova ProLlama 3-8B
ProviderAmazonMeta
Noometry Index31.025.5
Released2024-12-032024-04-18
WeightsProprietaryOpen
Context window300K—
Max output10K—
Input $ / M tokens$0.80—
Output $ / M tokens$3.20—
Results tracked3834

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Amazon Nova Pro leads

Amazon Nova Pro: 35.1 (#229), Llama 3-8B: 31.0 (#289)

BenchmarkAmazon Nova ProLlama 3-8B
LMArena Coding12701152
BigCodeBench Instruct—31.9%
LiveBench Coding38.1%—
BigCodeBench Complete—36.9%
HumanEval+—56.7%
MBPP+—54.8%

Agentic & Tool Use Not comparable

Amazon Nova Pro: 16.7 (#147), Llama 3-8B: —

BenchmarkAmazon Nova ProLlama 3-8B
Berkeley Function Calling Leaderboard25%—
TheAgentCompany1.7%—

Reasoning Amazon Nova Pro leads

Amazon Nova Pro: 20.0 (#243), Llama 3-8B: 14.3 (#326)

BenchmarkAmazon Nova ProLlama 3-8B
LMArena Hard Prompts12461133
Epoch Capabilities Index123.8116.45
Chess Puzzles—0%
LiveBench Reasoning32.6%—
DTBench—43.9%
LiveBench Data Analysis48.3%—
Adversarial NLI—57.3%
ForecastBench—58.6
LiveBench43.5%—
WinoGrande—75.7%

Math Amazon Nova Pro leads

Amazon Nova Pro: 28.5 (#243), Llama 3-8B: 8.8 (#323)

BenchmarkAmazon Nova ProLlama 3-8B
LMArena Math12521151
OTIS Mock AIME 2024-2025—1.9%
Omni-MATH24.2%—
LiveBench Math38%—
MATH Level 5—6.1%

Knowledge Amazon Nova Pro leads

Amazon Nova Pro: 27.4 (#250), Llama 3-8B: 7.8 (#308)

BenchmarkAmazon Nova ProLlama 3-8B
LMArena Expert12111113
MMLU82%68.8%
GPQA Diamond—26.1%
Humanity's Last Exam4.4%—
MMLU-Pro67.3%—
Confabulations30.1%—
Vectara Hallucination Rate5.1%—
GPQA (HELM)44.6%—
ARC (AI2) Challenge—82.8%
OpenBookQA—82.6%
TriviaQA—67.7%

Multimodal Not comparable

Amazon Nova Pro: 25.0 (#126), Llama 3-8B: —

BenchmarkAmazon Nova ProLlama 3-8B
LMArena Vision980—

Multilingual Amazon Nova Pro leads

Amazon Nova Pro: 39.7 (#223), Llama 3-8B: 30.8 (#261)

BenchmarkAmazon Nova ProLlama 3-8B
LMArena Non-English12341098
LMArena Chinese12441076
LMArena French12711159
LMArena German12431104
LMArena Japanese1200967
LMArena Korean12031004
LMArena Russian12401109
LMArena Spanish11821173

Instruction Following Amazon Nova Pro leads

Amazon Nova Pro: 64.9 (#226), Llama 3-8B: 58.4 (#260)

BenchmarkAmazon Nova ProLlama 3-8B
LMArena Instruction Following12351127
LiveBench Instruction Following67.1%—
IFEval81.5%—

Long Context Amazon Nova Pro leads

Amazon Nova Pro: 38.1 (#205), Llama 3-8B: 34.2 (#251)

BenchmarkAmazon Nova ProLlama 3-8B
LMArena Longer Query12551128

Writing & Preference Amazon Nova Pro leads

Amazon Nova Pro: 43.9 (#226), Llama 3-8B: 37.5 (#256)

BenchmarkAmazon Nova ProLlama 3-8B
LMArena Text12591166
LMArena Creative Writing12121150
LMArena Multi-Turn12461152
Short-Story Creative Writing60.5%—
WildBench77.7%—
LiveBench Language37%—

Frequently asked questions

Is Amazon Nova Pro better than Llama 3-8B?

Amazon Nova Pro is the stronger model overall, scoring 31.0 to 25.5 on the Noometry Index.

Is Amazon Nova Pro or Llama 3-8B better for coding?

Amazon Nova Pro scores higher on coding benchmarks: 35.1 versus 31.0 in the Noometry coding category.

How many benchmarks do Amazon Nova Pro and Llama 3-8B share?

19 benchmarks have published results for both models. Amazon Nova Pro has 38 scored results on Noometry and Llama 3-8B has 34.

Related comparisons

Go deeper