Noometry

Model comparison

Gemini 2.0 Pro vs Llama 3.1-405B

Gemini 2.0 Pro is the stronger model overall, scoring 38.6 to 30.4 on the Noometry Index.

Last verified . 4 shared benchmarks.

Gemini 2.0 Pro Google

38.6

Rank #171 Confirmed

Llama 3.1-405B Meta

30.4

Rank #279 Confirmed

Summary

  • They share 4 benchmarks with published results for both. Gemini 2.0 Pro scores higher in 6 categories and Llama 3.1-405B in 1 category; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Gemini 2.0 Pro leads 37.1 to 17.3.
  • The biggest single-benchmark swing is MATH Level 5: 83.5% for Gemini 2.0 Pro and 49.8% for Llama 3.1-405B.
  • Llama 3.1-405B has downloadable open weights; the other is API-only.

Side by side

Gemini 2.0 ProLlama 3.1-405B
ProviderGoogleMeta
Noometry Index38.630.4
Released2025-02-052024-07-23
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked1442

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 2.0 Pro leads

Gemini 2.0 Pro: 37.4 (#182), Llama 3.1-405B: 32.7 (#257)

BenchmarkGemini 2.0 ProLlama 3.1-405B
Aider Polyglot35.6%—
WeirdML—21.4%
LiveBench Coding63.5%—
LMArena Coding—1291

Agentic & Tool Use Not comparable

Gemini 2.0 Pro: —, Llama 3.1-405B: 20.9 (#131)

BenchmarkGemini 2.0 ProLlama 3.1-405B
TheAgentCompany—7.4%
Cybench—7.5%

Reasoning Gemini 2.0 Pro leads

Gemini 2.0 Pro: 21.6 (#194), Llama 3.1-405B: 16.4 (#293)

BenchmarkGemini 2.0 ProLlama 3.1-405B
Epoch Capabilities Index135.06128.76
SimpleBench—23%
Kagi LLM Benchmark—45%
EnigmaEval0.7%—
LiveBench Reasoning60.1%—
LMArena Hard Prompts—1270
DTBench—61.4%
LiveBench Data Analysis68%—
BIG-Bench Hard—82.9%
ForecastBench—59.9
HellaSwag—89.2%
LiveBench65.1%—
PIQA—85.9%
WinoGrande—89.2%

Math Gemini 2.0 Pro leads

Gemini 2.0 Pro: 37.1 (#101), Llama 3.1-405B: 17.3 (#283)

BenchmarkGemini 2.0 ProLlama 3.1-405B
MATH Level 583.5%49.8%
OTIS Mock AIME 2024-2025—9.7%
Omni-MATH—24.9%
LiveBench Math71%—
LMArena Math—1281

Knowledge Gemini 2.0 Pro leads

Gemini 2.0 Pro: 36.5 (#156), Llama 3.1-405B: 30.4 (#221)

BenchmarkGemini 2.0 ProLlama 3.1-405B
GPQA Diamond65.7%50.9%
Confabulations18.4%17.6%
MMLU-Pro—72.3%
GPQA (HELM)—52.2%
LMArena Expert—1243
ARC (AI2) Challenge—95.3%
MMLU—84.5%
TriviaQA—82.7%

Multilingual Not comparable

Gemini 2.0 Pro: —, Llama 3.1-405B: 40.9 (#209)

BenchmarkGemini 2.0 ProLlama 3.1-405B
LMArena Non-English—1248
LMArena Chinese—1242
LMArena French—1278
LMArena German—1252
LMArena Japanese—1208
LMArena Korean—1184
LMArena Russian—1265
LMArena Spanish—1259

Instruction Following Gemini 2.0 Pro leads

Gemini 2.0 Pro: 75.6 (#57), Llama 3.1-405B: 66.1 (#209)

BenchmarkGemini 2.0 ProLlama 3.1-405B
LiveBench Instruction Following83.4%—
IFEval—81.1%
LMArena Instruction Following—1259

Long Context Llama 3.1-405B leads

Gemini 2.0 Pro: 29.1 (#287), Llama 3.1-405B: 38.4 (#192)

BenchmarkGemini 2.0 ProLlama 3.1-405B
Fiction.LiveBench41.7%—
LMArena Longer Query—1266

Writing & Preference Gemini 2.0 Pro leads

Gemini 2.0 Pro: 52.7 (#164), Llama 3.1-405B: 39.1 (#246)

BenchmarkGemini 2.0 ProLlama 3.1-405B
LMArena Text—1284
LMArena Creative Writing—1262
EQ-Bench Creative Writing—870
WildBench—78.3%
LMArena Multi-Turn—1297
LiveBench Language44.9%—

Frequently asked questions

Is Gemini 2.0 Pro better than Llama 3.1-405B?

Gemini 2.0 Pro is the stronger model overall, scoring 38.6 to 30.4 on the Noometry Index.

Is Gemini 2.0 Pro or Llama 3.1-405B better for coding?

Gemini 2.0 Pro scores higher on coding benchmarks: 37.4 versus 32.7 in the Noometry coding category.

How many benchmarks do Gemini 2.0 Pro and Llama 3.1-405B share?

4 benchmarks have published results for both models. Gemini 2.0 Pro has 14 scored results on Noometry and Llama 3.1-405B has 42.

Related comparisons

Go deeper