Noometry

Model comparison

Amazon Nova Pro vs Gemma 1.1 7b IT

Amazon Nova Pro and Gemma 1.1 7b IT score almost the same on the Noometry Index (31.0 vs 31.3), so choose on price, context window or the category you care about most.

Last verified . 17 shared benchmarks.

Amazon Nova Pro Amazon

31.0

Rank #281 Confirmed

Gemma 1.1 7b IT Google

31.3

Rank #277 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Amazon Nova Pro scores higher in 5 categories and Gemma 1.1 7b IT in 3 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Amazon Nova Pro leads 43.9 to 30.4.
  • Gemma 1.1 7b IT has downloadable open weights; the other is API-only.

Side by side

Amazon Nova ProGemma 1.1 7b IT
ProviderAmazonGoogle
Noometry Index31.031.3
Released2024-12-03—
WeightsProprietaryOpen
Context window300K—
Max output10K—
Input $ / M tokens$0.80—
Output $ / M tokens$3.20—
Results tracked3819

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Amazon Nova Pro leads

Amazon Nova Pro: 35.1 (#229), Gemma 1.1 7b IT: 31.5 (#284)

BenchmarkAmazon Nova ProGemma 1.1 7b IT
LMArena Coding12701084
LiveBench Coding38.1%—
HumanEval+—35.4%
MBPP+—45%

Agentic & Tool Use Not comparable

Amazon Nova Pro: 16.7 (#147), Gemma 1.1 7b IT: —

BenchmarkAmazon Nova ProGemma 1.1 7b IT
Berkeley Function Calling Leaderboard25%—
TheAgentCompany1.7%—

Reasoning Too close to call

Amazon Nova Pro: 20.0 (#243), Gemma 1.1 7b IT: 20.5 (#238)

BenchmarkAmazon Nova ProGemma 1.1 7b IT
LMArena Hard Prompts12461071
LiveBench Reasoning32.6%—
LiveBench Data Analysis48.3%—
Epoch Capabilities Index123.8—
LiveBench43.5%—

Math Gemma 1.1 7b IT leads

Amazon Nova Pro: 28.5 (#243), Gemma 1.1 7b IT: 32.0 (#220)

BenchmarkAmazon Nova ProGemma 1.1 7b IT
LMArena Math12521107
Omni-MATH24.2%—
LiveBench Math38%—

Knowledge Too close to call

Amazon Nova Pro: 27.4 (#250), Gemma 1.1 7b IT: 28.3 (#247)

BenchmarkAmazon Nova ProGemma 1.1 7b IT
LMArena Expert12111039
Humanity's Last Exam4.4%—
MMLU-Pro67.3%—
Confabulations30.1%—
Vectara Hallucination Rate5.1%—
GPQA (HELM)44.6%—
MMLU82%—

Multimodal Not comparable

Amazon Nova Pro: 25.0 (#126), Gemma 1.1 7b IT: —

BenchmarkAmazon Nova ProGemma 1.1 7b IT
LMArena Vision980—

Multilingual Amazon Nova Pro leads

Amazon Nova Pro: 39.7 (#223), Gemma 1.1 7b IT: 28.1 (#273)

BenchmarkAmazon Nova ProGemma 1.1 7b IT
LMArena Non-English12341052
LMArena Chinese12441061
LMArena French12711065
LMArena German12431054
LMArena Japanese1200971
LMArena Korean1203988
LMArena Russian12401046
LMArena Spanish11821049

Instruction Following Amazon Nova Pro leads

Amazon Nova Pro: 64.9 (#226), Gemma 1.1 7b IT: 54.0 (#283)

BenchmarkAmazon Nova ProGemma 1.1 7b IT
LMArena Instruction Following12351057
LiveBench Instruction Following67.1%—
IFEval81.5%—

Long Context Amazon Nova Pro leads

Amazon Nova Pro: 38.1 (#205), Gemma 1.1 7b IT: 32.1 (#272)

BenchmarkAmazon Nova ProGemma 1.1 7b IT
LMArena Longer Query12551056

Writing & Preference Amazon Nova Pro leads

Amazon Nova Pro: 43.9 (#226), Gemma 1.1 7b IT: 30.4 (#288)

BenchmarkAmazon Nova ProGemma 1.1 7b IT
LMArena Text12591094
LMArena Creative Writing12121060
LMArena Multi-Turn12461040
Short-Story Creative Writing60.5%—
WildBench77.7%—
LiveBench Language37%—

Frequently asked questions

Is Amazon Nova Pro better than Gemma 1.1 7b IT?

Amazon Nova Pro and Gemma 1.1 7b IT score almost the same on the Noometry Index (31.0 vs 31.3), so choose on price, context window or the category you care about most.

Is Amazon Nova Pro or Gemma 1.1 7b IT better for coding?

Amazon Nova Pro scores higher on coding benchmarks: 35.1 versus 31.5 in the Noometry coding category.

How many benchmarks do Amazon Nova Pro and Gemma 1.1 7b IT share?

17 benchmarks have published results for both models. Amazon Nova Pro has 38 scored results on Noometry and Gemma 1.1 7b IT has 19.

Related comparisons

Go deeper