Noometry

Model comparison

Gemini 1.5 Flash (May 2024) vs Nemotron 3 Super

Nemotron 3 Super is the stronger model overall, scoring 40.1 to 33.2 on the Noometry Index.

Last verified . 16 shared benchmarks.

Gemini 1.5 Flash (May 2024) Google

33.2

Rank #246 Confirmed

Nemotron 3 Super NVIDIA

40.1

Rank #153 Confirmed

Summary

  • They share 16 benchmarks with published results for both. Gemini 1.5 Flash (May 2024) scores higher in 1 category and Nemotron 3 Super in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Nemotron 3 Super leads 39.6 to 22.1.
  • The biggest single-benchmark swing is WeirdML: 24.9% for Gemini 1.5 Flash (May 2024) and 38% for Nemotron 3 Super.
  • Nemotron 3 Super has downloadable open weights; the other is API-only.

Side by side

Gemini 1.5 Flash (May 2024)Nemotron 3 Super
ProviderGoogleNVIDIA
Noometry Index33.240.1
Released2024-05-142026-03-11
WeightsProprietaryOpen
Context window—262K
Max output—131K
Input $ / M tokens—$0.08
Output $ / M tokens—$0.45
Results tracked4221

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Nemotron 3 Super leads

Gemini 1.5 Flash (May 2024): 34.4 (#236), Nemotron 3 Super: 39.4 (#158)

BenchmarkGemini 1.5 Flash (May 2024)Nemotron 3 Super
WeirdML24.9%38%
LMArena Coding12611403
SciCode—36%
BigCodeBench Instruct43.5%—
BigCodeBench Complete55.1%—
ALE-Bench—213.9
HumanEval+75.6%—
MBPP+67.5%—

Agentic & Tool Use Not comparable

Gemini 1.5 Flash (May 2024): 26.6 (#102), Nemotron 3 Super: —

BenchmarkGemini 1.5 Flash (May 2024)Nemotron 3 Super
BALROG14.6%—

Reasoning Gemini 1.5 Flash (May 2024) leads

Gemini 1.5 Flash (May 2024): 21.7 (#215), Nemotron 3 Super: 18.7 (#276)

BenchmarkGemini 1.5 Flash (May 2024)Nemotron 3 Super
LMArena Hard Prompts12571386
NYT Connections (extended)—15.4%
CritPt—3.1%
DTBench53.8%—
Epoch Capabilities Index129.36—
ForecastBench53.9—
PIQA87.5%—

Math Nemotron 3 Super leads

Gemini 1.5 Flash (May 2024): 22.1 (#281), Nemotron 3 Super: 39.6 (#101)

BenchmarkGemini 1.5 Flash (May 2024)Nemotron 3 Super
LMArena Math12691380
MathArena Final-Answer Competitions—60.4%
OTIS Mock AIME 2024-202516.3%—
Omni-MATH30.4%—
MATH Level 561.9%—
FrontierMath (Feb 2025 set)0%—
GSM8K82.4%—

Knowledge Nemotron 3 Super leads

Gemini 1.5 Flash (May 2024): 26.2 (#260), Nemotron 3 Super: 38.9 (#139)

BenchmarkGemini 1.5 Flash (May 2024)Nemotron 3 Super
LMArena Expert12331398
GPQA Diamond47.3%—
MMLU-Pro67.8%—
GPQA (HELM)43.7%—
BoolQ85.8%—
MMLU77.9%—

Multimodal Not comparable

Gemini 1.5 Flash (May 2024): 36.0 (#81), Nemotron 3 Super: —

BenchmarkGemini 1.5 Flash (May 2024)Nemotron 3 Super
LMArena Vision1141—
Video-MME70.3%—
GeoBench76%—

Multilingual Nemotron 3 Super leads

Gemini 1.5 Flash (May 2024): 42.9 (#189), Nemotron 3 Super: 48.4 (#144)

BenchmarkGemini 1.5 Flash (May 2024)Nemotron 3 Super
LMArena Non-English12781355
LMArena Chinese12951432
LMArena French12581405
LMArena German12621341
LMArena Russian12881336
LMArena Spanish12431417
LMArena Japanese1252—
LMArena Korean1221—

Instruction Following Nemotron 3 Super leads

Gemini 1.5 Flash (May 2024): 66.8 (#205), Nemotron 3 Super: 71.2 (#154)

BenchmarkGemini 1.5 Flash (May 2024)Nemotron 3 Super
LMArena Instruction Following12581347
IFEval83.1%—

Long Context Nemotron 3 Super leads

Gemini 1.5 Flash (May 2024): 39.0 (#187), Nemotron 3 Super: 41.5 (#139)

BenchmarkGemini 1.5 Flash (May 2024)Nemotron 3 Super
LMArena Longer Query12841362

Writing & Preference Nemotron 3 Super leads

Gemini 1.5 Flash (May 2024): 48.7 (#196), Nemotron 3 Super: 55.9 (#140)

BenchmarkGemini 1.5 Flash (May 2024)Nemotron 3 Super
LMArena Text12871378
LMArena Creative Writing12851317
LMArena Multi-Turn12531369
WildBench79.2%—

Frequently asked questions

Is Gemini 1.5 Flash (May 2024) better than Nemotron 3 Super?

Nemotron 3 Super is the stronger model overall, scoring 40.1 to 33.2 on the Noometry Index.

Is Gemini 1.5 Flash (May 2024) or Nemotron 3 Super better for coding?

Nemotron 3 Super scores higher on coding benchmarks: 39.4 versus 34.4 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash (May 2024) and Nemotron 3 Super share?

16 benchmarks have published results for both models. Gemini 1.5 Flash (May 2024) has 42 scored results on Noometry and Nemotron 3 Super has 21.

Related comparisons

Go deeper